WooAh Choi
Machine Learning Engineer | AI Safety and Evaluation
Machine Learning Engineer with a production engineering foundation, working on AI safety and the evaluation of systems that are harder to verify than they look. MSc AI at Queen Mary University of London, where my dissertation applied mechanistic interpretability to a PPO-trained Transformer policy in partially observable RL. Sole author of a technical report on control boundaries for autonomous research agents. Previously built enterprise search and NLP in production, including the Supreme Court of Korea's e-litigation search.
About Me
MSc AI at Queen Mary University of London, with three years of industry experience as a Software Engineer at Konan Technology building enterprise search and NLP systems. My coursework spans Machine Learning, Neural Networks and NLP, Information Retrieval, Reinforcement Learning, and Conversational Agents — building depth in both theory and implementation.
Most recently I built ResearchAgentLab, an autonomous research agent, and then a control architecture for it after watching it silently substitute a different experiment for the one I had approved. Protocol Before Code freezes approved intent in a machine-checkable protocol, constrains how much the model is allowed to generate, and checks scientific conformance independently of software correctness. In holdout and fault-injection evaluations, 0 of 18 drifted protocols passed and 18 of 18 behaviour-changing trusted-component mutations were blocked before execution; with the content-integrity check neutralised, 15 of those same 18 escaped the remaining stack. A frozen 192-pair oversight experiment found that adversarial curation moved an overseer's target selection from 13.0% to 27.1% while explicit approval stayed at 100% in both conditions. Written up as a sole-authored technical report.
My dissertation is a mechanistic analysis of a PPO-trained causal Transformer policy in a partially observable MiniGrid T-maze. Its central finding was methodological: the aggregate success metric hid a failure mode, because an agent that always turns the same way scores around 50% while having learned nothing about the instruction cue. I designed a worst-case metric, Smin = min(S_key, S_ball), that exposes those one-sided policies, and used causal interventions to test control rather than infer it from correlation. Difference-vector attribution assigned 33–58% of the retrieval difference to padding, yet exchanging the padding source reversed no decision while exchanging the real-transition source reversed both — correlation and causation pointing at different things on the same policy. That signature held in only three of six checkpoints, so equal behaviour did not imply equal mechanism. A parameter-matched LSTM also solved the task at Smin = 0.898, which moved the contribution from architecture comparison to the analysis and evaluation framework itself.
Third author on a short paper accepted to FinNLP 2026 at EMNLP, on whether relevance-ranked web evidence is actually sufficient to verify financial claims. I was one of the two first-pass annotators, labelling 500 of the 750 claim–passage pairs under a blinded four-class protocol, including the 250-pair double-annotated block behind the paper's Cohen's kappa. I also review for the same workshop, the annual workshop of the ACL Special Interest Group on Financial and Economic NLP.
As co-founder of GridFlow Trade (selected for QMUL QIncubator), I built the UK day-ahead price forecasting product on 65,000+ real half-hourly settlement periods. The work I would point at is not the model. Early accuracy looked strong and was wrong: training showed MAE £7.4 while the same model served in production sat at £21.9, because training and inference were not seeing the same features. I rebuilt it as a single point-in-time feature pipeline shared by both paths and verified they produce identical features. The honest figure afterwards is MAE £24.2/MWh against a £34.2 naive baseline, a 29% error reduction on a 90-day held-out window averaged over five seeds. Benchmarking Naive, Ridge, LightGBM, XGBoost and a bidirectional LSTM on one leak-free split showed the tree models and Ridge were indistinguishable within seed noise, so the gains came from feature and evaluation design rather than model choice. I also audited 3.7 years of stored market data against the Elexon API and found a collector date bug that had corrupted 1,634 price and 1,634 system-margin records on every British Summer Time day.
At Konan Technology, I delivered production search and NLP systems for the Supreme Court of Korea, reducing litigation search latency from 5+ minutes to 8 seconds, and shipped query intelligence features across four enterprise clients.
Kaggle: Top ~17% in Google AI4Code · Top ~28% in RSNA 2022 Cervical Spine Fracture Detection.
Machine Learning
PyTorch, transformers and reinforcement learning — from fine-tuning BERT for production NER to training a causal Transformer policy with PPO
Search & Retrieval
Indexing at 100M+ scale, ranking and scoring logic, query understanding and expansion — the retrieval layer the current wave of AI is built on
Evaluation & Interpretability
Worst-case metrics, causal ablations and failure typologies — telling apart systems that work from systems that only score well
Production Engineering
Java, Spring and Python services, data pipelines, Oracle and PostgreSQL at scale, Docker and CI/CD on AWS — models are worth what they are worth in production
Work Experience
Konan Technology
Software Engineer, AI Solution Development Team
Nov 2020 — Jun 2023
Built and improved enterprise search and NLP systems across four clients, cutting Supreme Court litigation search latency from 5+ minutes to 8 seconds. Applied BERT-based NER, SFX/SRL query analysis, fastText, and vector search; shipped query intelligence features including related terms, recommendations, popular queries, and typo correction. Alongside the four builds below, I supported 14 shorter engagements across finance, government, healthcare, manufacturing, energy and policing — engine installs and integrations, escalation support, and internal training on the search stack. These included a redundant, high-availability search deployment for the Export-Import Bank of Korea, where the topology the client specified was not achievable under the constraints in place; I prepared the technical case, presented it, and we agreed an alternative that met the availability requirement.
Supreme Court of Korea - E-litigation Search and Analysis
Built a next-generation electronic litigation search and analysis system, reducing search latency from 5+ minutes to 8 seconds and resolving 375+ misanalysed cases. Optimised Java backend algorithms and indexing pipeline to achieve 37× search speed improvement. Applied speech act analysis (SFX) and semantic role labeling (SRL) to derive structured query constraints (time, location, case context). Fine-tuned a BERT-based NER model to extract case numbers, law names, and party names, with 4%+ improvement in extraction quality. Built vector search for related case retrieval and search analytics dashboards (line, bubble, dendrogram charts) for administrators.
Millie's Library - Book Search and User Pattern Analysis
Engineered an automated indexing system handling 100M+ data points, designed to scale beyond 1B records. Improved search ranking quality by 38% via custom scoring logic. Implemented auto-completion, related-term suggestion, and typo correction. Delivered time-series analysis of search terms for recommended and popular keyword features.
Heungkuk Fire & Marine Insurance - Main Website Search Renewal
Full-stack development of the web application's search system. Implemented AJAX-based continuous scrolling for smooth user experience and built auto-completion and popular search term features. Supported production rollout with log-based quality monitoring.
Ministry of SMEs and Startups - Venture Confirmation System Search
Developed search engine and RESTful API with text extraction and indexing from documents via Aspose and PDFBox. Implemented recursive file/folder deletion logic for efficient data management.
Skills
Select a skill to see the work behind it. Counts show how many projects and client deliveries match; add more to narrow.
NLP & Search
ML/AI
Data
Languages
Infrastructure
Backend
All work · 16
Supreme Court of Korea - E-litigation Search and Analysis
Built a next-generation electronic litigation search and analysis system, reducing search latency from 5+ minutes to 8 seconds and resolving 375+ misanalysed cases. Optimised Java backend algorithms and indexing pipeline to achieve 37× search speed improvement. Applied speech act analysis (SFX) and semantic role labeling (SRL) to derive structured query constraints (time, location, case context). Fine-tuned a BERT-based NER model to extract case numbers, law names, and party names, with 4%+ improvement in extraction quality. Built vector search for related case retrieval and search analytics dashboards (line, bubble, dendrogram charts) for administrators.
Millie's Library - Book Search and User Pattern Analysis
Engineered an automated indexing system handling 100M+ data points, designed to scale beyond 1B records. Improved search ranking quality by 38% via custom scoring logic. Implemented auto-completion, related-term suggestion, and typo correction. Delivered time-series analysis of search terms for recommended and popular keyword features.
Heungkuk Fire & Marine Insurance - Main Website Search Renewal
Full-stack development of the web application's search system. Implemented AJAX-based continuous scrolling for smooth user experience and built auto-completion and popular search term features. Supported production rollout with log-based quality monitoring.
Ministry of SMEs and Startups - Venture Confirmation System Search
Developed search engine and RESTful API with text extraction and indexing from documents via Aspose and PDFBox. Implemented recursive file/folder deletion logic for efficient data management.
ResearchAgentLab: Control Boundaries for Autonomous Research Agents
An autonomous research agent, and a control architecture built for it after it silently ran a different experiment from the one that had been approved. Protocol Before Code freezes approved scientific intent in a machine-checkable protocol, binds trusted components by content hash, limits how much the model is allowed to generate, and checks scientific conformance independently of software correctness. Stress-tested with generator-blind holdouts and fault injection.
Dissertation: Causal Cue Retrieval and Greedy Route Execution — A Mechanistic Analysis of a Transformer Policy
Mechanistic interpretability of a PPO-trained causal Transformer in a partially observable MiniGrid T-maze. Designed a worst-case cue metric that exposed one-sided shortcut policies hidden by aggregate success, then used causal interventions to verify genuine cue-dependent branch control rather than inferring it from correlation.
Multi-Domain Neural Search Engine — Domain Routing, Hybrid Retrieval and RAG Grounding Evaluation
A six-domain search engine with a domain-specialised BERT encoder behind each domain, a fine-tuned DistilBERT query router, hybrid BM25 and FAISS-HNSW retrieval fused with RRF, and ColBERTv2 late-interaction reranking. A RAG grounding layer was then added on top to answer a question the retrieval metrics cannot: whether a generated answer actually depends on the passages that were retrieved, or on what the model already knew.
GridFlow Trade — AI-Powered BESS Trading Platform
A production ML system for UK day-ahead power prices and battery trading, running the whole loop: live ingestion, validation, training, a versioned registry, batch and API serving, drift and service monitoring, automatic rollback or gated retraining, and walk-forward backtesting. The turning point was finding that the reported accuracy had been inflated by target leakage and train/serve skew, and rebuilding it as a single point-in-time feature pipeline shared by training and inference. The honest figure is MAE £24.2/MWh against a £34.2 naive baseline, 29% lower on a 90-day held-out window and 23% lower across a 12-month walk-forward. Public read-only demo on the live site. Selected for QMUL QIncubator.
Stock AI YouTube Pipeline
Automated weekly AI stock analysis video pipeline using yfinance, OpenAI GPT-4, moviepy, and Stable Diffusion. Fetches market data, generates narrated scripts, synthesises visuals, and uploads to YouTube end-to-end.
BuildU — AI Writing and Application Dashboard
"Build You" — a personal AI dashboard for job applications and English writing. Six tools behind one login: CV feedback with tracked results, an AI-text humaniser, a word-level text diff, IELTS writing practice with band-level scoring, and browser speech-to-text. Next.js 15 on Vercel, GPT-4 for the analysis, Tesseract.js for OCR.
Invntz Hackathon — E-commerce Aggregator
Web scraping and aggregation platform collecting product data from 5 e-commerce sites via automated browser crawling. Built with Spring Boot, Selenium, and Jsoup. Awarded 2nd place.
Generative AI Storytelling & Illustration Agent
End-to-end pipeline that converts keyword inputs into narrative stories using LLM APIs, then generates picture-book style illustrations via image-generation APIs. Extended to image-to-video workflows for animated output.
[ATE] Recipe & Ingredients Management
Full-stack application for managing recipes and tracking kitchen inventory, generating a shopping list from the gap between the two. I designed the data model and built the NestJS/TypeORM backend and its REST API against a Figma prototype. The project stalled when the front-end collaborator left mid-way with the login integration unfinished, so it never reached a usable build. The backend work stands; I intend to finish the client side myself.
SO Easy — Shared Office Booking Platform
Team project: platform connecting shared office providers with businesses seeking flexible workspaces. My contributions: search engine implementation and kanban board feature. Includes availability, booking management, and payment integration.
[Kaggle] Google AI4Code — Notebook Cell Ordering
Predicting the next cell to be executed based on the comments and code in Python Jupyter notebooks. Ensemble of CodeBERT and DistilBERT with different weights (0.748 / 0.252) based on the results of running each model.
[Kaggle] RSNA 2022 Cervical Spine Fracture Detection
Predicting the probability of a fracture in each of the seven cervical vertebrae. EDA, translation and summary of the EfficientNetV2 paper, data preprocessing, model training, and inference.
Featured Projects
ResearchAgentLab: Control Boundaries for Autonomous Research Agents
An autonomous research agent, and a control architecture built for it after it silently ran a different experiment from the one that had been approved. Protocol Before Code freezes approved scientific intent in a machine-checkable protocol, binds trusted components by content hash, limits how much the model is allowed to generate, and checks scientific conformance independently of software correctness. Stress-tested with generator-blind holdouts and fault injection.
Dissertation: Causal Cue Retrieval and Greedy Route Execution — A Mechanistic Analysis of a Transformer Policy
Mechanistic interpretability of a PPO-trained causal Transformer in a partially observable MiniGrid T-maze. Designed a worst-case cue metric that exposed one-sided shortcut policies hidden by aggregate success, then used causal interventions to verify genuine cue-dependent branch control rather than inferring it from correlation.
Multi-Domain Neural Search Engine — Domain Routing, Hybrid Retrieval and RAG Grounding Evaluation
A six-domain search engine with a domain-specialised BERT encoder behind each domain, a fine-tuned DistilBERT query router, hybrid BM25 and FAISS-HNSW retrieval fused with RRF, and ColBERTv2 late-interaction reranking. A RAG grounding layer was then added on top to answer a question the retrieval metrics cannot: whether a generated answer actually depends on the passages that were retrieved, or on what the model already knew.
GridFlow Trade — AI-Powered BESS Trading Platform
A production ML system for UK day-ahead power prices and battery trading, running the whole loop: live ingestion, validation, training, a versioned registry, batch and API serving, drift and service monitoring, automatic rollback or gated retraining, and walk-forward backtesting. The turning point was finding that the reported accuracy had been inflated by target leakage and train/serve skew, and rebuilding it as a single point-in-time feature pipeline shared by training and inference. The honest figure is MAE £24.2/MWh against a £34.2 naive baseline, 29% lower on a 90-day held-out window and 23% lower across a 12-month walk-forward. Public read-only demo on the live site. Selected for QMUL QIncubator.
Stock AI YouTube Pipeline
Automated weekly AI stock analysis video pipeline using yfinance, OpenAI GPT-4, moviepy, and Stable Diffusion. Fetches market data, generates narrated scripts, synthesises visuals, and uploads to YouTube end-to-end.
BuildU — AI Writing and Application Dashboard
"Build You" — a personal AI dashboard for job applications and English writing. Six tools behind one login: CV feedback with tracked results, an AI-text humaniser, a word-level text diff, IELTS writing practice with band-level scoring, and browser speech-to-text. Next.js 15 on Vercel, GPT-4 for the analysis, Tesseract.js for OCR.
Invntz Hackathon — E-commerce Aggregator
Web scraping and aggregation platform collecting product data from 5 e-commerce sites via automated browser crawling. Built with Spring Boot, Selenium, and Jsoup. Awarded 2nd place.
Generative AI Storytelling & Illustration Agent
End-to-end pipeline that converts keyword inputs into narrative stories using LLM APIs, then generates picture-book style illustrations via image-generation APIs. Extended to image-to-video workflows for animated output.
[ATE] Recipe & Ingredients Management
Full-stack application for managing recipes and tracking kitchen inventory, generating a shopping list from the gap between the two. I designed the data model and built the NestJS/TypeORM backend and its REST API against a Figma prototype. The project stalled when the front-end collaborator left mid-way with the login integration unfinished, so it never reached a usable build. The backend work stands; I intend to finish the client side myself.
SO Easy — Shared Office Booking Platform
Team project: platform connecting shared office providers with businesses seeking flexible workspaces. My contributions: search engine implementation and kanban board feature. Includes availability, booking management, and payment integration.
[Kaggle] Google AI4Code — Notebook Cell Ordering
Predicting the next cell to be executed based on the comments and code in Python Jupyter notebooks. Ensemble of CodeBERT and DistilBERT with different weights (0.748 / 0.252) based on the results of running each model.
[Kaggle] RSNA 2022 Cervical Spine Fracture Detection
Predicting the probability of a fracture in each of the seven cervical vertebrae. EDA, translation and summary of the EfficientNetV2 paper, data preprocessing, model training, and inference.
Latest Blog Posts
Writing about backend development, system design, and engineering practices
[Grid Flow Trade] When the price feed stopped: a postmortem
On 3 October the market index feed GridFlow forecasts stopped for nine hours, then fell behind six more times. Nothing false was stored, but the outag...
Logistic Regression(Linear Classification)
1. Introduction to Linear Classification In the field of machine learning, supervised learning represents a class of applications where algorithms lea...
IELTS Writing Task2 Template2 (Agree vs Disagree)
# Intro There is a disagreement regarding the issue of whether or not _. Even though there are people who think that _, I personally think that it is ...
IELTS Writing Task2 Template1 (Agree vs Disagree)
# Intro There is a disagreement about the issue of A and B I partially agree that A 현상 can sometimes be more important then B현상 because A B 결과 요약 # Ag...
화상회화 Day1(2023.03.19) 정리
Free talking feedback 동사를 잘 쓰는데 주어를 많이 빼먹는 편 provide라는 단어를 많이 반복해서 사용하는 버릇이 있음. feedback 주는 것은 provide보다 suggest가 더 어울림 "I didn't anything” → "I didn...