Unexpected "error_no_members" when load testing cowboy server

[Kozaky]

Hi,

I just began a simple cowboy rest service using the pooler configuration defined in the readme (basically setting up the basic config for the pool and adding it to the project as a "included_applications"). The rest service only consist of a simple endpoint where I query a database:

hello_to_json(Req, State) ->
    case pooler:take_member(pg_db, {1, min}) of
        error_no_members ->
            Body = jsone:encode(#{error => <<"could not get a database connection">>}),
            io:fwrite("Error!!~n"),
            {Body, Req, State};
        Conn ->
            {ok, _, [{Id, Name, Address, _, City}]} = epgsql:squery(Conn, "select * from person"),
            Body =
                jsone:encode(#{id => Id,
                               name => Name,
                               address => Address,
                               city => City}),
            {Body, Req, State}
    end.

My understanding is that, with the current approach, the process should wait for a minute before returning a "error_no_members". That's the case when I try it in the shell; however, when I load test the service with wrk (wrk -t4 -c100 -d30s http://127.0.0.1:3000/), the service begins to get "error_no_members" a lot of times even though the waiting time is longer than the load test is running.

Thanks in advance!

[seriyps]

Could it be because you don't explicitly return member to the pool with pooler:return_member?

Also, would be cool if you share your pool's config.

[seriyps]

Also, it would help if you log results of

pooler:pool_stats(pg_db),
pooler:pool_utilization(pg_db),

in the error_no_members clause.

[Kozaky]

Hi!

Thanks a lot for the prompt response. I tried adding the pooler:return_member but it did not solve the issue.

Regarding the pool config, this is my current setup:

% pooler app config
[
 {pooler, [
         {pools, [
                  #{name => pg_db,
                    max_count => 5,
                    init_count => 2,
                    start_mfa =>
                     {epgsql, connect, [#{ host => "localhost", username => "postgres", password => "pass", database => "postgres", timeout => 4000}]}}
                 ]}
           %% if you want to enable metrics, set this to a module with
           %% an API conformant to the folsom_metrics module.
           %% If this config is missing, then no metrics are sent.
           %% {metrics_module, folsom_metrics}
        ]}
].

This is the current implementation after following your advise:

hello_to_json(Req, State) ->
    case pooler:take_member(pg_db, {1, min}) of
        Conn when is_pid(Conn) ->
            {ok, _, [{Id, Name, Address, _, City}]} = epgsql:squery(Conn, "select * from person"),
            Body =
                jsone:encode(#{id => Id,
                               name => Name,
                               address => Address,
                               city => City}),
            pooler:return_member(pg_db, Conn),
            {Body, Req, State};
        error_no_members ->
            Body = jsone:encode(#{error => <<"could not get a database connection">>}),
            io:fwrite("Error!!~n"),
            io:format("stats: ~p~n", [pooler:pool_stats(pg_db)]),
            io:format("utilization: ~p~n", [pooler:pool_utilization(pg_db)]),
            {Body, Req, State}
    end.

These are the logs where you can find the stats and utilization:
logs.txt