PHPBrowser module: Wrong url sent and request does not match codeception debug output

Reproduce la prueba seeAvatarInfo mostrada con la URL de PhpBrowser establecida primero en http://web y después en http://web:80. Comienza con el código de resolución de URL de Symfony BrowserKit Client.php al que se hace referencia y con la configuración de PhpBrowser/REST. Se considera terminado cuando una URL relativa después de una solicitud previa se envía al host configurado sin la ruta de historial errónea, con una prueba de regresión que cubra el caso sin puerto.

What are you trying to achieve?

Assert that a file can be retrieved from a webserver using the REST module with PHPBrowser, by checking if the status code is equal to 200. The file exists on the server and can be retrieved using CURL with the same url as shown in the debug output of codeception.

What do you get instead?

Failed asserting that 404 matches expected 200 and wrong debug output of codeception.

Root cause

I've managed to successfully reproduce it and I think I've tracked down the issue:
My setup consists of docker images (nginx image proxying to php-fpm image) defined in a docker-compose.yml file as service web and app respectively. Codeception is run in separate docker image attached to the same network.

The problem seems to be the PHPBrowser url parameter as defined in the suite config file shown below. When there is no port defined and the url is http://web (the name of the nginx host since it listens to port 80) it sends a completely wrong url to the webserver. Instead of sending web/storage/avatars/... as described in the debug output of codeception the actual url received by nginx is me/web/storage/avatars/... You can see nginx logs below for more details. Now where did that me came from..??

It turns out that if you define a port in the PHPBrowser url variable, such as http://web:80 everything works as expected and the test passes. If you don't, like described above and for the specific test case I have posted below the url gets messed up and looks to be resolved from History urls.
Looking at the codeception src code the offending method seems to be in vendors symfony/browser-kit: https://github.com/symfony/browser-kit/blob/c55fe9257003b2d95c0211b3f6941e8dfd26dffd/Client.php#L322.

Looking at that method, specifically here: https://github.com/symfony/browser-kit/blob/c55fe9257003b2d95c0211b3f6941e8dfd26dffd/Client.php#L593 it seems that it tries to resolve the url based on History urls and since for my specific testcase I already have a previous http request sent in the same test, the url gets wrongly constructed.

Workaround

The workaround is, as shown above, to always define the port even if it is 80 or 443. I don't understand why the url needs to be constructed using previous history and I cannot say I agree with that approach but I also cannot suggest a fix at the moment.

Details

Provide console output if related. Use -vvv mode for more details.

Codeception output:

  [Page] /me/avatar
  [Response] 200
  [Request Cookies] []
  [Response Headers] {"Server":["nginx/1.15.0"],"Content-Type":["application/json"],"Transfer-Encoding":["chunked"],"Connection":["keep-alive"],"X-Powered-By":["PHP/7.2.8"],"Cache-Control":["no-cache, private"],"Date":["Thu, 09 Aug 2018 08:00:18 GMT"],"X-RateLimit-Limit":["200"],"X-RateLimit-Remaining":["197"],"X-Frame-Options":["SAMEORIGIN"],"X-XSS-Protection":["1; mode=block"],"X-Content-Type-Options":["nosniff"]}
  [Response] {"status":200,"payload":{"url":"web/storage/avatars/0f74205c-75f5-42f5-852e-48b32fb87820_gr95QH7P5RMuyzzEWysQngzb54Pj5CHyma3K1Ffk.jpeg","created_at":{"date":"2018-08-09 08:00:18.000000","timezone_type":3,"timezone":"UTC"},"updated_at":{"date":"2018-08-09 08:00:18.000000","timezone_type":3,"timezone":"UTC"}}}
 I see response code is 200
 I see response is json 
 I see response json matches json path "$.payload.url"
 I grab data from response by json path "$.payload.url"
 I assert not empty "web/storage/avatars/0f74205c-75f5-42f5-852e-48b32fb87820_gr95QH7P5RMuyzzEWysQngzb54Pj5CHyma3K1Ffk.jpeg"
 I send head "web/storage/avatars/0f74205c-75f5-42f5-852e-48b32fb87820_gr95QH7P5RMuyzzEWysQngzb54Pj5CHyma3K1Ffk.jpeg"
  [Request] HEAD web/storage/avatars/0f74205c-75f5-42f5-852e-48b32fb87820_gr95QH7P5RMuyzzEWysQngzb54Pj5CHyma3K1Ffk.jpeg []
  [Request Headers] {"Authorization":"Bearer token_reducted"}
  [Page] web/storage/avatars/0f74205c-75f5-42f5-852e-48b32fb87820_gr95QH7P5RMuyzzEWysQngzb54Pj5CHyma3K1Ffk.jpeg
  [Response] 404
  [Request Cookies] []
  [Response Headers] {"Server":["nginx/1.15.0"],"Content-Type":["application/json"],"Connection":["keep-alive"],"X-Powered-By":["PHP/7.2.8"],"Cache-Control":["no-cache, private"],"Date":["Thu, 09 Aug 2018 08:00:18 GMT"]}
  [Response] 
 I see response code is 200
 FAIL 

Nginx access log:

web_1     | 172.27.0.4 - - [09/Aug/2018:08:15:40 +0000] "GET /me/avatar HTTP/1.1" 200 324 "-" "Symfony BrowserKit"
web_1     | 172.27.0.4 - - [09/Aug/2018:08:15:40 +0000] "HEAD /me/web/storage/avatars/ec20bfa9-c252-43bb-b990-f97362445ec2_jBqWIZuRm62qJsXz1sAwfrrMvCzeps8ZXsrKxWZp.jpeg HTTP/1.1" 404 0 "http://web/me/avatar" "Symfony BrowserKit"

Provide test source code if related

    public function seeAvatarInfo(\AcceptanceTester $I)
    {
        $I->haveHttpHeader('Authorization', 'Bearer ' . $this->token);

        $I->sendGET('/me/avatar');
        $I->seeResponseCodeIs(200);
        $I->seeResponseIsJson();
        $I->seeResponseJsonMatchesJsonPath('$.payload.url');
        $url = $I->grabDataFromResponseByJsonPath('$.payload.url')[0];
        $I->assertNotEmpty($url);

        $I->sendHEAD($url);
        $I->seeResponseCodeIs(200);
    }

Details

• Codeception version: 2.4.5
• PHP Version: 7.2.8
• Operating System: Docker (official php-fpm images) in OSX
• Installation type: Docker (codeception image)
• Suite configuration:

class_name: AcceptanceTester
modules:
    enabled:
        - Asserts
        - PhpBrowser:
            url: 'http://%APP_URL%'
        - REST:
            depends: PhpBrowser