Dot matrix with data pipelines

Santhosh Muthukrishnan

Software Engineer — AI & Cloud Pipelines

Do you use data in your business? I can help!

I build LLM and multimodal applications. I have delivered serverless cloud architectures, fine-tuned LLMs, and turned stakeholder requirements into deployed AI solutions.

Currently looking for data engineering, agentic AI and LLM projects worth building.

Stats

Lambda speed-up
12x
Compute cost
−60%
Cloud billing, weighted avg
−2%
Best Gleason accuracy
82.56%
Models benchmarked
12
MSc result
81.8%
Years in industry
3

Experience

4 roles

AI Research Assistant

May 2026 – Aug 2026

Maynooth University · Maynooth, Ireland

  • Designed multi-source data ingestion pipelines on a GPU-accelerated HPC cluster, using Slurm for scheduling, queuing and resource allocation across parallel training runs.
  • Built preprocessing for multi-GB Whole Slide Images, with patching strategies that make gigapixel inputs tractable for training.
  • Benchmarked 8 transformer models and 4 vision-language models across 3 datasets; the best reached 82.56% accuracy on patch-wise prostate cancer Gleason score classification.
  • Extended the research pipeline into a diagnostic web app where users upload WSI/patch inputs and pick from 12 trained models.
  • Engineered a reproducible experiment framework — data and library versioning, config management, logging — for rapid iteration.

Research Intern

May 2025 – Dec 2025

ADAPT Centre · Dublin, Ireland

  • Built multimodal pipelines fusing tabular clinical data with 3D MRI imaging, including 3D → 2D conversions to fit model input constraints.
  • Engineered around real-world clinical data problems: sparse longitudinal records, missing values, misaligned timestamps and schema mismatches.
  • Built and validated an XGBoost pipeline end to end, reaching R² = 66.4%.
  • Used auxiliary language models to generate structured annotations at scale and fed them into downstream training.

Software Engineer

Jan 2022 – Jun 2024

Codemantra · Chennai, India

  • Optimised PDF-processing Lambda workloads by up to 12x, eliminating 15-minute timeouts and cutting compute costs by 60%.
  • Owned backend microservices and cloud applications in an AI-powered Intelligent Document Processing product for PDF accessibility.
  • Built APIs (Spring Boot, Flask) and serverless infrastructure (Lambda, SAM, SNS, SQS) for dataset enumeration and model inference.
  • Built the annotation tooling clients used to produce labelled image–JSON pairs for training document-structure models.
  • Automated internal workarounds that lifted ticket activity and closure by 30%.

Java Developer

Sep 2021 – Dec 2021

Dockabl · Bengaluru, India

  • Built REST APIs and fixed bugs for an HR analytics platform, and introduced Java Checkstyle rules to improve code quality.

Projects

4 entries

Biomedical Literature RAG System

  • MedCPT
  • FAISS
  • BM25
  • Hugging Face
  • Torch
  • Gemini Flash

A Retrieval-Augmented Generation QA system over PubMed abstracts that fuses BM25 and MedCPT dense retrieval with Reciprocal Rank Fusion and a cross-encoder reranker; Gemini Flash generates answers that cite the retrieved abstracts by PMID. Evaluated on 5,729 BioASQ questions, hybrid retrieval raised Recall@10 from 0.655 to 0.683 and MRR@10 from 0.859 to 0.869 over a BM25 baseline, and reranking pushed Recall@10 to 0.699. Deployed on Hugging Face Spaces (ZeroGPU) with end-to-end p50/p95 latency of 2.3 s / 4.7 s: retrieval runs in 21 ms and reranking 100 candidates in 273 ms at p95, with LLM generation about 90% of latency. Truncating reranker input to 256 tokens halved it per candidate at about 1% relative recall loss. The static frontend is on Cloudflare.

Ongoing

Cloud-Native PDF Asset Extraction System

  • AWS Lambda
  • Amazon S3
  • DynamoDB
  • REST APIs
  • Python
  • Java

An event-driven AWS pipeline for extracting images, captions and metadata from PDFs — Lambda for processing, DynamoDB for text, S3 for assets — built for large-scale ingestion, indexing and retrieval.

Discontinued

Properly accessible PDFs with good image captions are now very hard to find, which left the pipeline without the source data it was designed for.

OpenCodeReasoning (debugging focus)

  • Qwen 2.5
  • Hugging Face

A challenge to improve an LLM's coding and logical reasoning using NVIDIA's OpenCodeReasoning dataset — coding problems, buggy code and critiques — to help the model catch and fix bugs.

Discontinued

A loosely specified challenge built on LLM-generated critiques and code. There was no reliable way to measure accuracy, so I couldn't tell whether anything was actually improving.

YTBUDDY: YouTube Shorts pipeline

  • Google App Engine
  • Google Drive API
  • YouTube Data API
  • OAuth 2.0
  • Python
  • Flask

A web app that takes long videos (streams, vlogs) from Google Drive, cuts them into 30–90 second clips and uploads them to YouTube as Shorts, with a dashboard for choosing videos, approving clips and scheduling uploads.

Discontinued

The YouTube API cost money to use at the time, and with free tools already doing the same job, there was no case for selling it to clients.

Skills

7 groups

Data infrastructure
  • AWS Lambda
  • SAM
  • SNS
  • SQS
  • Event-driven processing
  • Dataset & annotation tooling
  • Batch & async workflows
Databases
  • PostgreSQL
  • MongoDB
  • DynamoDB
  • Redis
Cloud & DevOps
  • AWS (Serverless, EC2, S3, RDS)
  • GCP
  • Git
  • Docker
  • Kubernetes
  • Jenkins
  • GitHub Actions
Backend
  • Java
  • Spring Boot
  • Spring MVC
  • Python
  • Flask
  • Node.js
  • Microservices
  • JWT
  • Hibernate
AI / ML
  • NLP
  • LLMs & VLMs
  • Slurm HPC
  • Hugging Face
  • RAG
  • LangChain
Data analysis
  • R
  • MATLAB
  • PostGIS
  • scikit-learn
  • pandas
  • NumPy
Testing
  • JUnit
  • Mockito
  • Selenium
  • Cypress

Education

2 degrees

MSc Data Science and Analytics

2024 – 2025

Maynooth University · Maynooth, Ireland

First-class honours (81.8%)

Machine Learning, Statistical Methods, Geospatial Analysis; thesis on vision-language models in medical settings.

BTech Computer Science and Engineering

2018 – 2022

Vellore Institute of Technology · Vellore, India

First-class honours (87.5%)

Software Engineering, Data Structures and Algorithms, Data Analytics, Cloud Computing.

Let's work together

A role, a project, or just an idea worth discussing — send me a message and I'll get back to you.