In a medium saucepan, combine the sugar, flour, and salt. Gradually whisk in the milk until smooth.

Place the saucepan over medium heat and cook, stirring constantly, until the mixture thickens and begins to bubble, about 8-10 minutes.

In a small bowl, lightly beat the egg yolks. Slowly add about 1/2 cup of the hot milk mixture to the egg yolks while whisking to temper them, ensuring they don't scramble.

Pour the tempered egg yolk mixture back into the saucepan, stirring constantly. Cook for an additional 2-3 minutes until the pudding is thickened.

Remove the saucepan from heat and stir in the vanilla extract.

Let the pudding cool slightly, stirring occasionally to prevent a skin from forming.

In a serving dish or trifle bowl, layer the vanilla wafers, banana slices, and pudding. Start with a layer of wafers, then bananas, and then pudding. Repeat until all ingredients are used, ending with pudding on top.

Cover the dish with plastic wrap and refrigerate for at least 4 hours or overnight to set and allow the flavors to meld.

Before serving, optionally top with whipped cream and a few crushed vanilla wafers for garnish.