PostgreSQL does not highly prioritize making the act of connecting to the database one that happens quickly. On each connection, PostgreSQL creates a new process to talk to the client. This is a very expensive operation because the operating system creates a new process and copies all the data. Things like the pg_hba.conf authentication implementation are optimized for security and flexibility, even if it's at the expense of speed. The presumption is that users will run things for a long time relative to how long it takes to connect.
When this isn't true, which can be the case for web applications in particular, connection pooling is one approach to reducing this overhead. The connection pool sits between your application and the database. It makes a fixed number of connections to the database, typically under 100, and keeps them open all the time. As incoming requests come in, those connections in the pool are re-used. The DISCARD ALL command can be used in a PostgreSQL session to make it fresh for a new connection. When clients disconnect, the pooler resets the session without dropping the database connection, leaving it ready for a new connection to use.