Trying to understand usage of Mojo::UserAgent inside of a Mojolicious app

[dklawren]

Maybe there is something in the inner workings off Mojolicious that I do not yet understand but I cannot seem to figure out how to call a route from the same app using Mojo::UserAgent inside the app. If I use Mojo::UA to call out to an external site it works fine. I have a large app that uses Mojo (bugzilla.mozilla.org) but I have created a test case in Mojo::Lite for demostration.

use Mojolicious::Lite;
use Mojo::UserAgent;

get '/foo' => sub {
  my $c   = shift;
  my $resp  = Mojo::UserAgent->new()->get('http://localhost:3000/test');
  $c->render(text => 'Value for foo is ' . $resp->res->json->{foo});
};

get '/test' => sub {
  my $c = shift;
  $c->render(status => 200, json => {foo => 'bar'});
};

app->start;

If I start the above the the call times out and tries several times.

$ perl test_mojo_lite.pl daemon
[2021-03-24 17:57:17.51314] [1531] [info] Listening at "http://*:3000"
Web application available at http://127.0.0.1:3000
[2021-03-24 17:57:31.20464] [1531] [debug] [V0qBwei3] GET "/foo"
[2021-03-24 17:57:31.20492] [1531] [debug] [V0qBwei3] Routing to a callback
[2021-03-24 17:58:11.24720] [1531] [error] [V0qBwei3] Can't use an undefined value as a HASH reference at test_mojo_lite.pl line 13.
[2021-03-24 17:58:11.24744] [1531] [debug] [V0qBwei3] Template "exception.development.html.ep" not found
[2021-03-24 17:58:11.24756] [1531] [debug] [V0qBwei3] Template "exception.html.ep" not found
[2021-03-24 17:58:11.24764] [1531] [debug] [V0qBwei3] Rendering cached template "mojo/debug.html.ep"
[2021-03-24 17:58:11.25036] [1531] [debug] [V0qBwei3] 500 Internal Server Error (40.045699s, 0.025/s)
[2021-03-24 17:58:11.25072] [1531] [debug] Inactivity timeout
[2021-03-24 17:58:11.25187] [1531] [debug] [M2ACYqJD] GET "/test"
[2021-03-24 17:58:11.25237] [1531] [debug] [M2ACYqJD] Routing to a callback
[2021-03-24 17:58:11.25271] [1531] [debug] [M2ACYqJD] 200 OK (0.000815s, 1226.994/s)
[2021-03-24 17:58:11.25787] [1531] [debug] [5G3i6wv7] GET "/foo"
[2021-03-24 17:58:11.25818] [1531] [debug] [5G3i6wv7] Routing to a callback

If instead of using http://localhost:3000 and just use /test in the Mojo::UA get() then it works instantly.

  my $resp  = Mojo::UserAgent->new()->get('/test');
  $c->render(text => 'Value for foo is ' . $resp->res->json->{foo});

If I use Mojo::UA and call out to an external API site not currently on localhost, it also works. What am I missing with regards to making a request to the current app running on localhost:3000 as if it were some far away site?

Another question somewhat related, If I wanted to do away with using Mojo::UA in the /foo route, how would i slurp in the result of the /test route and access the returned JSON data instead of using $c->redirect_to()?

Thanks!

[doojonio]

I suppose you started this app with morbo? Well, then your app is just one process. In this case, your user agent will wait until the time expires. After that, it will throw a 500 error and only then morbo will process the request that your user agent made.

Launch this app with prefork (multiple processes):

./your_mojo_app.pl prefork

[dklawren]

This actually made an improvement for me. My development environment was in fact using Morbo as it will pick up changes to the source code and reload automatically instead of having to rebuild the docker containers each time when I wanted to make a slight change and see it work or not.

[Grinnz]

Applications requesting from themselves is not supported. I suggest refactoring so that you can call the common code as a helper or method, or splitting up the application.

[dklawren]

The reason for this is writing some test code to test our OAuth2 client functionality and wanted to created some mocked API endpoints that we can put in the apps configuration and call those mocked endpoints locally instead of the actual OAuth2 provider.

[Grinnz]

Mock endpoints are usually best implemented in a test application (Lite apps can be inlined into test files or modules), rather than as part of the same application.

[rabbiveesh]

I agree wholly with @Grinnz that you should just factor out common code.

Other than that, switching get to get_p and getting the result from the resulting promise will allow the event loop to run, so your request can be handled.

[dklawren]

I will look at using promises although it will be a new thing for me so I did not try that before.

[dklawren]

I tried the above sample code using a get_p promise and I am not having much luck getting it to work.

use Mojolicious::Lite (-signatures);

use Mojo::UserAgent;

get '/foo' => sub {
  my $c  = shift;
  my $result;
  my $ua = Mojo::UserAgent->new;
  $ua->get_p('http://127.0.0.1/test')->then(
    sub ($tx) {
      $result = $tx->res->body;
    }
  )->catch(sub ($err) {
    warn "Connection error: $err";
  })->wait;

  $c->render(text => $result);
};

get '/test' => sub {
  my $c = shift;
  $c->render(text => 'Hello World!');
};

app->start;

perl test_mojo_lite.pl prefork
[2021-03-25 02:34:04.45692] [14920] [info] Listening at "http://*:3000"
Web application available at http://127.0.0.1:3000
[2021-03-25 02:34:06.86112] [14920] [debug] [rj6yvfRg] GET "/foo"
[2021-03-25 02:34:06.86142] [14920] [debug] [rj6yvfRg] Routing to a callback
[2021-03-25 02:34:06.86214] [14920] [debug] [rj6yvfRg] Template "foo.html.ep" not found
[2021-03-25 02:34:06.86274] [14920] [error] [rj6yvfRg] Could not render a response at test_mojo_lite.pl line 19.

[2021-03-25 02:34:06.86298] [14920] [debug] [rj6yvfRg] Template "exception.development.html.ep" not found
[2021-03-25 02:34:06.86322] [14920] [debug] [rj6yvfRg] Template "exception.html.ep" not found
[2021-03-25 02:34:06.86336] [14920] [debug] [rj6yvfRg] Rendering template "mojo/debug.html.ep"
[2021-03-25 02:34:06.87365] [14920] [debug] Your secret passphrase needs to be changed
[2021-03-25 02:34:06.87485] [14920] [debug] [rj6yvfRg] 500 Internal Server Error (0.013725s, 72.860/s)
Connection error: Premature connection close at test_mojo_lite.pl line 16.

Best I can tell the route /foo subroutine is returning early and Mojo is killing any pending connections/transactions before they can get a chance to complete (although it should happen quickly).

Pointers?

[rabbiveesh]

wait doesn't wait for the promise when the event loop is running.

You need to render the page from inside the then of the promise. If you want it to look like synchronous code, you could check out Future::AsyncAwait.

$ua​->​get_p(​'​http://127.0.0.1/test​'​)​->​then(

    ​ sub​ (​$tx​) {
      ​ $result​ = ​$tx​->​res​->​body;
$c->render(text => $result)
     }
   )​->​catch(​sub​ (​$err​) {
    ​ warn​ ​"​Connection error: ​$err​"​;
   })​;

Or

use Mojolicious::Lite -async_await;
Use Syntax::Keyword::Try;
...
try {
my $tx = await $ua​->​get_p('http://127.0.0.1/test​'​)​;
$c->render(text => $tx->res->body)
 } catch (​$err​) {
    ​warn​ ​"​Connection error: ​$err​"​;
 }

[rabbiveesh]

Remind me to not try writing code on my phone ever.

[jhthorsen]

The issue is that the $ua inside your blocking get() use $ua->ioloop, while the server uses Mojo::IOLoop->singleton. You can fix this by either using get_p or use the non-blocking version of get() (with a callback). The second solution would be to set $ua->ioloop(Mojo::IOLoop->singleton).

Here is another example that will work nomatter which port you start your app on:

my $c = shift;
my $ua = Mojo::UserAgent->new;
$ua->ioloop(Mojo::IOLoop->singleton)->server->app($c->app);
my $tx = $ua->get('/test');
$c->render(text => 'Value for foo is ' . $tx->res->json->{foo});

(I haven't run the code, so there might be dragons)