Start tracking everything. There are lots of phone apps out there that offer great, and easy ways of tracking your data.
Tournament poker is inherently a very high variance form of poker. The best in the world are not cashing, let alone winning, the majority of tournaments they enter. The appeal of tournaments is they offer a way to massively increase your number of buy-ins very quickly if you win. I think for the majority of people out there tournaments are a losing approach if you want to make money, but they are however often very fun. As you said bad players can get lucky, good players can get a string of unfortunate hands and due to the nature of increasing blinds in tournaments can easily bust even though they may have played very well.
Back to the point on getting better, as I said start tracking things. I really do mean tracking and being honest about it. Many apps make it very easy to list the buy-in, rebuys you did, tips gave, hours played, breaks, known players, interesting hands, etc. Poker is a game to play well you need a lot of honest reflection on your ability and a lot of work off the table. Once you're tracking things, there are lots of great places online where you can watch content and read it from the best. Vlogs, courses, forums, etc. Books are another option, especially for live poker and even more so for pub poker. Books are quickly outdated but that is only really true at the top of the game, I still find that books like Harrington on Hold'em, Raiser's Edge still have a lot of value in very weak games, which often pub leagues are. Most of the great books are not going to make you into a top pro in the modern game, but they will teach out how to play good fundamentals and often that is enough against pub and weak live players.
Something else I'd say is have a mindset shift. You want to have a growth mindset, especially if you're serious about improving. You mentioned that you played before and that you've improved, but how long ago was that, how honest are you being to yourself, are those learnings you had actually good fundamentals or not? I'm not saying that to be negative to you, but I've recently returned to playing regularly after a few years of only casual play. I essentially sat down and re-learned the game and tightened up my fundamentals and learned newer concepts in the game. A big part of that was being honest with myself that what was decent play in 2014/2015 is maybe not so great in 2023. Having that growth mindset means you approach things to learn and don't get frustrated.
In terms of measuring it as I said track everything you can. I think in poker some ways of measuring if someone is good or not can be very subjective. I think the most reliable way is win rate and hours played. Like if you have 1,000$ profit after 100 hours, or 10$ an hour, it's not really enough hours to say that you've been lucky or unlucky. Given a good live dealer will get maybe 25 hands per hours out. Then even if you're playing a very wide range of 25%, you've only played 625 hands over those 100 hours. Easiest way to see you're improving is your win rate increases or actually remains positive over thousands of hours.