Automate and simplify workflows and tasks with software solutions.
I have led and contributed to various software development and lifecycle management activities since starting my career at the Met Office in 2002. My main interests are in workflow automation and configuration management, for building and running mission critical scientific applications on HPC, Linux clusters, and the Cloud. I enjoy helping colleagues to reduce complexity in their software. My work is used in both research and production locally and by international partners. I have varying level of experience in a variety of programming languages including Python, Bash, Perl, Java, Fortran, C, C++, JavaScript and SQL. I am familiar with technology such as Docker, Conda, GitHub Actions and Sphinx.
Expert Scientific Software Engineer, Weather Science Research To Operations, Met Office.
Improving the user experience in managing and working with large workflow suites for the next generation modelling systems for research to operational use.
I am advising developers and users on how to modularise large model application configuration files, using a YAML processor, a Python project I created and maintain. I am contributing to C++ applications so developers can generate configuration schema files, which can be used to provide validation services and IDE integration.
I am developing the release/deployment process of the next generation data assimilation model applications that are built from many projects under active development by several partner organisations. I am looking for gaps in our process, as well as opportunities for automation.
I advise and help developers on how to prepare their Python projects for continuous integration and packaging for release/deployment.
Expert Scientific Software Engineer, Business Group Post Processing Applications, Met Office.
I led the development of a workflow suite to allow scientists to trial different new post-processing applications and techniques with many years of data. The success of this work allowed our scientists to study new ways to deliver new science capabilities to our customers.
I proposed and demonstrated how to migrate an existing post processing workflow from running on-premises to running in a serverless architecture on AWS, to reduce cost and increase flexibility. I developed a prototype that would automate building, infrastructure creation, and launching the workflow on the fly.
I removed bottlenecks in various application logic to enable on-demand API to trigger post processing applications in the backend, reducing the overall runtime from minutes to a few seconds.
Expert Scientific Software Engineer,
Senior Scientific Software Engineer,
NWP Applications Developer, Weather Science IT, Met Office.
I led the design, development, and lifecycle management of metomi/rose, a Python-based system to manage configuration for model application runtime. It provided our scientists with a common environment and a common set of toolkits to work on and manage their model configurations for robust reproducibility. The system has been in use operationally for many years.
I led and coordinated the development and lifecycle management of cylc/cylc-flow, a Python-based workflow scheduling system, which provided our scientists, engineers, and system analysts a common environment to run workflow suites to submit jobs on the HPC and other computing resources, from simple workflows to complex cycling workflows and from research to operation. This workflow scheduling system has been used by all Met Office core forecasting workflow suites for many years.
I led the design, development, and lifecycle management of metomi/fcm, a wrapper to Subversion and a Perl-based fully featured build system for modern Fortran applications. The latter provided our scientists with a very efficient environment to compile their code (compared to a traditional Makefile environment) and to reproduce build configurations. Its introduction has reduced build time of some of our largest science applications from hours to a few minutes. I wrote this system over a decade ago and it is still in use today for building our largest science model applications due to its efficiency and effectiveness.
I contributed to the design and development of MOOSE, the Java-based software layer of the Met Office massive archive system, providing users with a common interface to archive and access data. I contributed to the design of the client-server interaction and implemented the client logic. The software is still in production use after 15 years.
I created a Fortran application to ingest forecast model data from different weather forecasting centres. The data can come in different forms, with grids of the globe drawn in different ways and coming in as fragments at different time. The application re-integrates the fragments and converts the data to a format compliant to ours. The application is still in use after more than 15 years.
I ported software systems across several generations of HPC and Linux/Unix clusters, some of which have very different architectures and operating systems.
MEng (1st Class Honour) 1998 and DPhil (aka PhD) 2002, Material Science, Trinity College, University of Oxford.