Reading Thinking in Systems (Donella Meadows), especially "Places to Intervene in a System". I'm thinking about this in the context of software development (and problems in product teams). Wrote a first draft blog post.

Ultimately my bigger focus is on how systems in our world operate. That's the real work to be engaged in: how can we intervene in global culture and production and supply changes and ... ? Things are truly a mess. Last week I was looking at The Future is Degrowth. I definitely want to read that.

I've been studying MLOps, partly because I want to build and deploy systems myself, but also to understand how to develop datasets and models in the real world. AI product teams are different than software engineering. It's more experimental, but this can be focused for production with data validation tests and well organized tooling. I like being the bridge person to connect these.

I'm talking with a Smart Building / IoT company for a potential CTO or VP of Engineering role. I've been focused on architecture and engineering, and primarily I want to continue to work in Climate Tech / Sustainability, but I will definitely consider CTO/VP and dedicate myself to directing traffic and making the product -> development pipeline flow. A lot of this job would be architecture and planning how best to manage all the data. Fairly small company, but well funded to grow this year (just got Series A). Interviews continuing next week.

Studying IoT protocols and networking, ML for predictive maintenance.

Re-reading The Manager's Path, Patterns of Enterprise Application Architecture (Martin Fowler) (old but still relevant for the patterns).

Tested Hatch — it promises to be a single solution for Python project management. Lord knows we need that. It combines dependency management, virtual environments, multiple pythons, testing with a matrix (like tox) and publishing. It starts out nice and simple and it works. Gets a bit complicated when you have multiple environments. I need to look at it more before recommending it. Would it work well when building a Docker image?

In comparison, Poetry just focuses on dependency management and publishing.

They are both lovely.

Worked again on Sensilla — my "declarative, reactive web scraper" side project. It's been in development for far too long; I keep getting jobs and then it gets parked again for a year or more. I worked on this last month and then parked it again for vacation.

Sensilla scrapes web sites and twitter, builds a link graph and can export datasets and graph data. These will then be labelled and used to train a Multi-label classification model, NER, and other experiments.

I'm on a new M1 computer now, so I got new Docker containers running for the database (Postgres), reloaded from a dump and everything is back up and running. (When you run Docker on an M1 the image architectures change to arm.) No problems migrating at all.

For Sensilla I'm planning how to efficiently determine what needs to be fetched without having to just load everything from the db and compare sets. Considering Bloom filters and other probabilistic data structures, though at this stage it's not the way to do it.

Nice looking probabilistic filter Node library: callidon.github.io/bloom-filters