Key Projects
My personal favourites
Personal Tooling Platform
A full-stack web platform for launching and tracking asynchronous tools, with a Next.js frontend, Express API, PostgreSQL database, and Python workers that can run on separate hardware.
Why build this?
My current company does not allow us to run code on our computers for security reasons. This website lets me build predefined bots at home and orchestrate them from the internet with clean UI.
Architecture
- PostgreSQL as the central source of truth
- Next.js handles the frontend, SSR pages and client-side data fetching where appropriate
- Express provides an API layer where I can manage views and auth
- Python workers poll for jobs and process them asynchronously
- Workers are decoupled from the hosted web app, this allows the site to run light in the cloud and the heavier workloads can run on owned hardware
JavaScriptTypeScriptPythonHTML & CSSReactNext.jsTailwindExpressSQLAlchemySelenium
Link Checker
My first worker node inside of the above project - this allows people to inspect the link structure of their website
Why build this?
With SEO efforts, building a strong internal linking structure is useful for UX and for Google's rankings. Tracking which links work, where they are and how many of them there are is hard so this automates the process.
Architecture
- Poll jobs in Postgres
- When there is a new job, claim it and launch the scraping function
- Track pages in one table and the relation of links on another
- Iterate through all pages and you get a representation of the link structure for the entire website
PythonSQLAlchemySelenium