Posts tagged as programming

I recently shipped some client work - a small prototyping project - written in Python. Which is surprising, given I would say - if asked - that “I don’t write Python“.

A lot of people write a lot of Python these days. It’s a common teaching language; it’s a lingua franca for machine learning and data science; it’s used as a scripting language for products I use such as Kicad or Blender. But it’s passed me by. I first wrote Ruby in 2004, which I still love, and that’s served my needs as a general-purpose scripting language, as well as a language for building web applications (with Rails). These days, I’ll also write a lot of Typescript. I don’t really need another scripting language.

Given that “I don’t write Python“… why did I just ship a project in it, and how did I do that?

---

First of all: why Python for this project?

I needed something that could write to a framebuffer from a command-line, running in an event-driven style. I also needed something that could be used cross-platform, and wouldn’t be too hard for anyone else to pick up. Given this was a prototyping project, ease of manipulation and modification were important, so an interpreted language with no build step would be helpful. As would good library and community support. pygame looked like it would be a very good fit for our project.

Whilst I wouldn’t admit to writing Python, it turns out I had a fairly strong ambient knowledge of it already. I knew its basic syntax, and was used to its module-style imports via Typescript. I was well aware of its lack of braces and significant whitespace. And someone who loves functional-style code, I was already a big fan of was list comprehensions.

I paired that ambient knowledge with a couple of core programming skills.

Firstly : my experience of other event-driven languages and platforms (like Processing, p5js, and openFrameworks) meant I was already familiar with the structure the code would need. An imperative, event-driven structure is going to be pretty similar whether you’re writing in Processing or pygame; I tend to write these with as small a main loop as possible, and then core functions for “set up all the data for this frame” and “render this frame”, broken down as appropriate. You can smell the porting in my code, plus I’m sure I could be more idiomatic, but structure is structure; this meant we began with solid foundations.

Next: reading documentation. pygame has good documentation, and thanks to its maturity, there’s a lot of online resources for it - not to mention excellent resources for Python itself. That, combined with good experience of usefully parsing error message got us a long way.

Modern tooling helps a lot:

• language support in editors/IDEs has gotten really good; the Language Server Protocol means that developer support for interpreted languages is better than it’s ever been. More than just spicy syntax-highlighting, its detailed and comprehsnive support is exactly what you want as an “experienced, but not in this language” developer.
• “What about ChatGPT?“ This is not an article about how I wrote code without knowing how, all thanks to ChatGPT, I’m afraid. But one thing I have begun to occasionally use it for is as a rubberduck; a “Virtual Junior Developer.” “I’m looking for an equivalent to Ruby’s x in Python. Can you explain that to me, knowing that I’m an experienced Ruby developer?“. It usually won’t give me precisely what I’m looking for. But it will give me something in the ballpark that suddenly unblocks me - much like a good pairing partner might. Sometimes, that’s why I need.
• A rubbderduck I find much more useful is Github Copilot. Again, I don’t think its strength is the “write code for me” functionality. Instead, I most like its ability to provide smart, contextual autocomplete. That’s doubly useful in loosely typed languages, where language servers can be limited in their recommendations in a way that LLMs aren’t limited by. I’m particularly grateful for Copilot as a solo developer; it feels like a pairing partner chipping in with ideas, which, even when they’re not right, at least steer me in a new direction. I’ve found Copilot particularly speeds development up later on in a project. As the codebase gets larger, its intuition becomes more developed.

The combination of core competencies, modern tooling, and a well-established platform enabled us to not only motor through the initial porting process, but achieve our further goals, with space and time left for polish and improvement within the budget. The final codebase met the client’s needs: it worked well cross-platform, was straightforward (thanks to virtualenv) to get up and running on a colleague’s development computer, and is a good foundation for future work.

---

Programming work - I hesitate to say “creative programming” because all programming is creative - is about much more than just ‘knowing a language’, and this project was a great example of that.

Much more important is knowing how to learn new languages (and how you best learn new things); understanding patterns common to styles of language; maintaining good code hygiene and great documentation; and finding ways to steer yourself in the right direction - whether that’s a partner to pair with, the right developer tool, a good online resource, or even a modern application of AI.

My evaluation at the beginning of the project had been validated. As I’d suspected, actively choosing to work in a new-to-me platform was not a risk. It turned out to be exactly the right tool for the job - a prototype where speed of iteration and flexibility were highly valued.

That evaluation comes down to something I often describe as a sense of smell; a combination of expertise, personal taste, and ‘vibes’. Perhaps it’s easier just to call that experience. It turns out that “twenty years of writing software” beats the fact that almost none of it has been Python.

We wrapped this small project last week, and I enjoyed my exposure to Python as a more experienced developer. I also valued being reminded me of what experience feels like, and what it enables: good judgment, flexibility, and focusing on the project’s outcomes, rather than getting lost in implementation.

I’ve spent much of 2022 working with SvelteKit as one of my primary development environments. Kit hit its 1.0 release in December 2022. I’ve spent some time evangelising it to developer colleagues, many of whom may not have encountered it, and a year with Kit feels like a good time to summarise that time, and explain why I’m spending so much time with it.

First, we should start with Svelte.

Svelte

Svelte is a reactive component framework for building web front-ends. Think of it like React or Vue. To the eye, it resembles Vue much more than React: it uses single-file components, broken into script, markup, and styles. Unlike Vue, it uses a templating language with specific tags, rather than ‘special’ attributes on HTML elements.

It’s quite straightforward, in that it doesn’t do a great deal. It provides a way of building components, passing data to them, and reacting to changes in data. It has some unusual syntax, but throughout, it tries to ensure all its syntax is valid Javascript syntax.

Throughout the entire project, it’s clear that where possible, it wants to fit into the web as it is, using standards where it can, and syntax or patterns developers know.

There are couple of things that might surprise you coming from other frameworks.

Firstly: reactivity uses two-way binding. Obviously, this is an easy shortcut to a footgun, and might sound smelly or unsafe to you. But Svelte makes it safe, and it leads to terser syntax and makes it easier to write code that states your intent in a literate way, rather than dancing around one-way React-style binding.

Secondly - and this is partly why that two-way binding is safe: Svelte is a compiler. When you visit a site that uses it, you’re not downloading all of Svelte to use the page; Svelte compiles the site down to only the script needed to make it run. And in doing so, throws out all the Svelte-specific syntax or code, makes sure you’re not doing anything impossible. It emits compact, fast code.

(This is all because, thirdly, Svelte is really a language)

If that’s Svelte, what is SvelteKit (or “Kit”) for short?

SvelteKit

SvelteKit is to Svelte as Next is to React, or Nuxt is to Vue. It’s a full-stack, isomorphic framework. That means: you write your code once, but it could run on the front-end or back-end.

By default, when you hit a URL, the server renders that page, and serves it to you - and then future navigation uses the History API and requests just JSON from the backend, rendering the front-end reactively. The documentation doesn’t really use the phrase “full-stack”, and perhaps that term has served its purpose. The technology skews towards the front-end, as most site frameworks built on top of component frameworks do - but it’s definitely everything you need to make a site. The answer to “but where does it run” is “yes”.

In a Kit site, the front-end is made of Svelte components - all pages are, in fact, themselves just Svelte components - and the back-end is Javascript. As well as a routing system for displaying components, Kit offers ways of building server-side only backends, loading code to pull data from the back-end, and building the site to run on a range of deployment environments (more on this later).

At this point I imagine somebody sucking their teeth and having opinions about Single-Page Apps (I sure do). But SvelteKit is much more interesting than that.

SvelteKit sites build using an adapter system. Adapters transform your development code to code suited for production deployment in a particular environment. The same code can be deployed to different infrastructures with different adapters.

In development, you work against an Express server, and you could make a production build that uses this approach using adapter-node, emitting an app you can just run with node index.js in your build directory. Put it onto Heroku, or behind an nginx proxy, and off you go.

Or maybe you just want an entirely static site - you have no dynamic code at all, and could bake the whole thing out. adapter-static will let you do this.Calculate the routes you want to prerender, and build the whole site to a directory you can host on S3 or Apache. You can even tear out all the rehydration and build a site that has no Javascript in it at all.

But most interesting are the adapters for modern “JAMstack” hosts, like Netlify or Vercel. Build a site with adapter-netlify and all your static pages will get prerendered as before… and your dynamic pages, or back-end endpoints, will all get turned into (eg) Netlify Functions, to run on Lambdas as appropriate.

(I like this because it pushes back against one of my issues with JAMStack sites: you control the front-end, and when it comes to anything resembling a back-end, you just give somebody else your credit card details and use their API. Kit makes it easier for front-end-preferring developers to write their own tiny parts of back-end code, integrated into modern front-end-preferring hosting platforms. Obviously you’ll still need some kind of storage, but I like that you’re not having to jump between cloud/lambda functions in one place, and front-end code in another: your application should be all one thing, and you should write as much of it as you can).

Whilst there are always going to be host-specific issues, this somewhat decouples application design from application build; a kind of future-proofing (not to mention the ability to change your mind later, as a static site develops more dynamic features). You can even throw away all the static pages and dynamic routing and make a classic SPA that lives only in the client, if you really want to.

Standards all the way down

But what I enjoy most is how “of the web” it all feels. One of Kit’s key design philosophies is using standards when they exist. That means fetch throughout, polyfilled on the server, natively in the client; that means Request and Response objects on raw server endpoints; that means Form Actions that, by default, simply POST to an endpoint and redirect, before the documentation goes on to show you how the enhance directive can be used to start dynamically updating the page without a reload for those users who have scripts enabled. As a developer who likes his POST and GET, this is the way. A user on the discussion forums was trying to work out how to call server-side functions from the client, and I pointed out they just had to POST to their paths; I could feel their eyes bulge a little, before they then realised that perhaps this was how it always should be.

The tooling is particularly good - especially its use of static typing, and its VS Code extension. It’s lovely to return a typed object on the serverside, and have your front-end components pass that typing all the way down - or to raise an IDE error when they’re not quite getting what they expected.

And integrating SvelteKit with existing code and libraries is lovely, because Kit does so little. From the outside, it might not look like the ecosystem is nearly as big as React’s. But that doesn’t necessarily matter: bringing existing libraries into Svelte seems much easier than bringing them into React land. A colleague I’d introduced to SvelteKit confirmed this after doing exactly that on a project not two months after starting using it. “It turns out I didn’t need the libraries,“ he said, and just wrote his own integration with a handful couple of lines; a far cry from trying to find yet another plugin to useSomething.

One reason I often pick it is the familiarity of all the moving parts. There’s something about the syntax that makes it so easy and welcoming to pick up for newcomers. I pick Svelte often because I can get other developers who know HTML/CSS/JS|TS up and running on it really fast. It’s even more ‘natural’ to work in than Vue, and especially so than JSX/React.

And finally, the reason I feel confident in it is… I like the taste of its core team. Rich Harris - ex-New York Times, now at Vercel - is thoughtful, considered, and has what I think is good taste. When it comes to frameworks, taste is important: if you don’t agree with the core philosophy of the project, which inevitably means the taste of the project leads… you likely never will. But I do - that focus on straightforward tools to build moderate-sized products, focusing on One Way To Do Things, an emphasis on excellent documentation and (especially) interactive tutorials, and above all, something that feels more like the Web: standards all the way down.

I’ve used it recently on a few projects:

• a blog rewrite, handling thousands of pages across 10 or 12 template files; primarily focusing on static delivery of key pages, and then dynamically rendering the long tail of the site
• a content site that, on the content⟷software scale, was firmly at the ‘content’ edge, having been written in Jekyll. Rewriting it in Kit allowed us to move it a few notches towards software. It became easy to, in a mainly Markdown-driven site, still embed dynamic or interactive components inside Markdown articles - or reach out to external services for certain components based on page-level metadata. We began rendering this statically, and moved to using a dynamic renderer when the ability to serve it statically wasn’t quite as ready as we hoped.
• highly interactive prototypes of mobile UI and apps, using all the features of a modern browser (touch events, MediaRecorder and MediaStream APIs, gestures) to build a rich, interactive prototype.

It works well for all of these. In particular, its static-generation performance is… pretty competitive with some of the dedicated JS Static-Site Generators.

Mainly, though I like using a modern, isomorphic-style component-based web framework that still feels of the web, still interested in pages and URLs and HTTP and accessibility, whilst allowing me to use modern tooling and the reactive pattern. I also continue to love the enthusiasm for it that everyone I share it with has. It’s not my only hammer, nor is it the only thing I plan to use going forward (and I’m still going to reach for Rails when I’ve got CRUD and data-heavy applications) - but it’s been a great tool to work with this year, and I hope to share it with more developers in time, and work with it again.

On the projects front:

• Longridge is getting off to a good start. I’m hunting down contributors, doing introductions, firing off emails. I’m also beginning to write some scripts just to get my hand in and to see if they line up with what the production crew are expecting. Not quite where I wanted to be by the end of Friday, but the team seem pleased where we are, and I had several good meetings with them during the week so I’ll take that and carry that momentum into next week.
• Dent is going back to the drawing board, but possibly not for a month or so, given Longridge combined with impending holiday. It turns out the current build just won’t behave; I spent some time with a J-Link really prodding at the thing and no amount of work in gdb would solve my problems. So we’ll start again in October.

I spent a little time getting myself quickly up-to-speed with some new technologies this week - always good to keep my hand in, and at least one of them was an audition for Longridge:

• I finally wrapped my head properly, clearly around CSS Grid; gosh, what a breath of fresh air. It makes two-dimensional grid layouts just a breeze to write. I also managed to understand its relationship with (and differences to) flexbox. After a long period of reading docs, I now had understanding in my hands. Good.
• To that end, I managed to tear out all of Bootstrap from this site - which had used it for positioning. Replaced the lot with CSS Grid and Flexbox in under an hour. Very happy with that - and also happy that the trade-offs for users of older browsers are, to my mind, totally acceptable.
• I continue to poke and prod at Javascript frameworks, and spent an evening (off-duty) kicking the tyres on Svelte. I’m interested in Svelte for how it approaches the reactive-UI pattern, but also how it removes the load on the client by just… being a compiler. Given the Javascript community’s fondness for (or, perhaps, reliance on) transpilation, going the whole hog doesn’t seem that much of a difference. I quickly ported my personal ‘hello world’ of reactive UIs to it, and enjoyed the process, the clarity, and the light page load. I’m still not sure how I feel about the ‘magical’ binding and slightly higher need for simple stores; I found myself using stores and subcription in places React would force me to explicitly pass functions and state down, endlessly, which to my mind is both good and bad., Still, I like its approach to reactivity and its single-file component structures, and especially its approach to performance and webbishness.

And that was the technology I played with; not project-oriented, but useful for me in lots of ways, and worth sharing here, as a log of what I get up top.

A good week - but gosh, the next two are going to be busy.

I kicked off work on Longridge this week, with a remote meeting to go over the shape of the initial work, and some of the approach. That’s given me some grounding, some ideas for a deadline at the end of Week 342, and - most importantly - some homework before a workshop in week 343. That homework lead to some planning, research, and an afternoon of kicking tyres on the internet - amongst other things. I actually need to check what I can say about Longridge - I think I can be reasonably public - so I might describe that more next week.

I spent a day brushing up on the state of web development in 2019. I’m a lapsed front-end developer back from the days when that primarily involved deep knowledge of HTML and awful, awful browsers. (I definitely have built some IE6-compatible sites in my time). Since then, I’ve seen front-end change a lot, and done a nice pile of work on Captionhub with the HTML5 media extensions and spec. But there’s still new things to learn, so I spent an afternoon bringing myself up-to-speed with CSS Grid. Gosh, it’s good, isn’t it? It solves a problem elegantly, and still gives - in many situations - completely acceptable fallbacks when it’s not available. A really elegant API, and a nice bit of technology. I have a small backburner project that I’m using to learn new things on, and I spent some more time on that this week, too; it’s nearly hitting a nice alpha point, so perhaps a concerted day or two in the future will push that over the hill.

It’s nice catching up with technology once it’s a little established. I subscribed hugely to the notion of choosing boring technology. I’ve never been let down by boring technology, and, some days, it’s been reassuring to have my bacon saved by not picking something that goes out of fashion as fast as it came in, or doesn’t have support, or just ended up being the wrong horse. It’s why I still am entirely comfortable shipping Rails projects: it works, it’s expressive for developers, Performant Enough, and gets web-apps based around shipping content to/from databases over HTTP out the door quickly. Uncontroversial. So rather than hurtling to stay up-to-date with trends, I’m comfortable keeping one eye on them, and the other on the Unexciting Present. I’ll read, think, compare, but committing to using them is a very different process. Now that I’m in a lull, it’s time to catch up a little and explore.

I also shipped a few small pieces of code related to things I’ve been doing recently.

Firstly, some Javascript: wxr_to_json is a small command-line node tool for converting Wordpress eXtended Rss to JSON files, simply for ease of processing. It’s a little opinionated - it flattens some one-item arrays into objects - but it works very well for large WXR dumps; v8 and xml2js are very, very quick.

Secondly, some Ruby. I packaged up a gist by Stefan Daschek into a gem to use as a Capistrano plugin. It’s ideal for deploying static sites (including build processes) with cap: it makes a clean local checkout, gives you hooks to run build processes, and then uses rsync to move built content to a server (and lets you use all of cap’s versioned deployment tools). The code worked, but it felt cleaner to turn it into a gem, rather than a lib file floating around my repository. So I finished the bundling job and wrote a pile of documnetation. No idea if either of these will be useful to other people, but they’re easily shared. So let’s do that, then, and perhaps someone else will find them useful. (Why are you using a server and not a CDN for a stic site, you might ask? To avoid yakshaving, primarily. Change one thing at once!)

And that was a week.