Do fun software, games especially - you'll learn heaps.
Don't do open source for the sake of it, unless its a product you actually use.
Perhaps you could fork my simple raycaster and add your own enhancements to it, its a simple project [0]
[0] - https://github.com/con-dog/2.5D-raycasting-engine/blob/maste...
The only solution I found working is to leave data field and get back to software one (within same org or not).
No other reliable and meaningful way.
Nothing beats actual job in improving a complex skill. There is no shortcut to imitate that.
Could you elaborate on this a bit?
> But for a software engineer, the amount of programming feels low. It's more like creating quick scripts.
I feel like it’s the other way around. When I worked as a data analyst (before transitioning to a data scientist and eventually a full-fledged SWE), my experience was quite different. As a DA, I mostly wrote aggregation queries, built dashboards, and did some Python, Pandas, and Notebook munging.
On the other hand, SWE is a different world. I guess it depends on the type of SWE you are and the nature of the organization you work for. At a large-scale organization, as an SWE, I did far more programming work than I ever did as a DA.
That said, there are different ways to cope and build skills—it takes time. I do side projects and, more importantly, write about them regularly [1]. Since I have to publish something each week, it forces me to think about different problems and seek out new technologies to try out so I can write about them. I also write about things I’ve figured out at work, which has been immensely helpful. Over the years, this act of actively seeking knowledge has added up and become substantial.
More than once, people have reached out with job offers just because they stumbled across my writing on Hacker News or other social media platforms.
Your approach might be different. Find something tangential to programming and work on it outside of work hours. Even a few hours per week adds up. For me, it’s side projects and writing. For you, it might be something else.
[1]: https://rednafi.com
AI has made some once absurd ideas possible. For example, you can hardcode personal details and have a resume builder export it to PDF or a web portfolio. It'll probably be less work than formatting a resume in Word. Leave AI to do repetitive work like mapping data types, while you figure out the code to make it look pretty.
If you're looking for suggestions outside of work, one important factor will be motivation. You're not paid for it so you'll need something that will hold your attention regardless, or you won't commit to it. Therefore, the answer depends on what you personally find interesting.
As someone in a similar situation, my number one piece of advice is to meticulously keep up your software engineering best practices even when doing your day to day data analysis job. All that bread-and-butter stuff like tests and easy deployment and well structured code and reproducible builds and writing reusable tools and packages tends to fall by the wayside when your 'just' hacking one off Jupyter notebooks.
As a software engineer you bring a perspective to the job that many of your colleagues may lack. Lean on that and use that background to help build better tools and ways of working. This not only makes your job (and the job of your colleagues) easier in the long run it will also let you keep your software engineering skills sharp.
If you write a lot of "quick scripts", you might be writing a lot of the same boilerplate code over and over again. If that's the case, creating a framework/library that does X/Y/Z (related to your job) might help.
The Missing Semester of Your CS Education: https://missing.csail.mit.edu/ I would say finishing this helped me a lot to learn better and build up on tech stuff I use at my day job. This is a nice short course giving a very useful intro to software engineering tools required upon which you can build up further on the go, otherwise it's very easy to miss those topics and stay an 1x developer.
If you’re interested in security-related self-studies, I can warmly recommend Cryptopals: https://cryptopals.com/
Use them
I think one of the most toxic things in this industry is that we think we need to continuously be doing coding outside of work instead of living life.
So the answer is I don’t.
I had been programming continuously from the time I was in 6th grade in 1986 until 2020.
As I pivoted into more strategic cloud consulting specializing in cloud architecture + application development in 2020 - “cloud application modernization” - I started coding less and less to the point now that almost any time that I even suggest writing code at work even though my title is “staff software architect”, I get my hand slapped and asked why didn’t I delegate it and focus on “higher impact work”.
I keep my knowledge of the industry up to date by continuously learning during the work day. But when I get off work, I shut my computer down and live my life - spending time with my wife and friends, exercise, travel, concerts, etc.
We travel - a lot. One of the benefits of remote work.
If I do need to pivot back into hands on coding, I have a years worth of expenses in savings in the bank aside from retirement savings to give me time to get back into the groove of things.