I agree with your positions about short seasons and brand new big bads.

However, I don’t think TNG, and classic Trek at large, have a future totally devoid of “the pains and pitfalls of present-day life”. For instance, Captain Maxwell blows up a bunch of Cardassian outposts, and there was that whole incident with the Pegasus and the cloaking device. These are clear instances showing in TNG’s world, we haven’t completely grown out of the darker parts of our nature.

I think the ideal of Star Trek is there is a future where we have overcome many of our problems, and when new (or old, sometimes) arise, we can work together to overcome them and improve ourselves.

In some ways, I think that Lower Decks embodies this extremely well. Because it’s supposed to be a comedy, it liberates the show from a lot of modern sci-fi conventions; this allows a largely utopian environment for our Federation characters where they’re free to help each other evolve far beyond the borderline insane sitcom archetypes they started the show as.