Insurance is, at its core, a data business. Every pricing decision, every underwriting judgement, every fraud investigation and every claims settlement is fundamentally a data problem. Which is why the sector’s AI ambitions are so credible in theory and so difficult to execute in practice. The data is there. In most large insurers it is vast. The challenge is that it is fragmented, poorly structured, and sitting in legacy systems that were never designed to feed a modern machine learning pipeline.
To understand why this matters, it helps to understand what the most advanced use cases in insurance actually require. A modern fraud detection model in motor insurance does not just look at the claim in front of it. It cross-references the claimant’s history across multiple policies, analyses the language patterns in the claim narrative using natural language processing, checks vehicle data against telematics records, compares the repair estimate against a database of known inflated quotes from specific garages, and flags the output for investigation if the probability score crosses a threshold. Running that model in production requires clean, consistent, accessible data from at least five or six different source systems, a pipeline that ingests and processes it in near real-time, and an engineering team that can maintain it as the underlying systems change.
Most insurers do not have that infrastructure. And the teams needed to build it are in extremely short supply.
What the data function in insurance actually needs to look like
The shift to AI-led underwriting and claims is forcing a fundamental rethink of what a data function in an insurance business needs to look like. Traditionally, actuarial teams owned the data models and IT teams owned the infrastructure. That division worked well enough when the models were statistical and ran in batch overnight. It does not work when the expectation is real-time AI inference at the point of underwriting or claims triage.
What is emerging in the more advanced carriers is a genuinely integrated data engineering function: data engineers building and maintaining the pipelines, data scientists and ML engineers developing and validating the models, BI developers making the outputs accessible to underwriters and claims handlers, and platform engineers keeping the cloud infrastructure performant and compliant. Each of these is a distinct specialism. Each requires people who understand not just the technology but the insurance context it sits inside. A data engineer who has never worked in insurance needs significant time to understand actuarial data structures, Lloyd’s market data standards, or the regulatory requirements around how claims data can be used in pricing models.
The market for these people is fiercely competitive. Data engineers and ML engineers in financial services are being recruited by fintech firms, banks, and technology companies who can offer higher salaries, more modern environments, and clearer career paths. Insurance has historically struggled to attract early career data talent. According to recent research from the London Market Group, graduate hires across the city marketplace grew at just two percent annually between 2021 and 2023, against premium growth of around seven percent. The pipeline is not keeping up.
Why the standard approach is not working
The typical response has been to try to hire experienced data professionals from the open market, supplement them with consultancy resource on specific projects, and run training programmes for existing staff. Each of those approaches has real limitations. The open market for senior data talent is expensive and the supply is thin. Consultancy resource delivers a project but does not leave capability behind when it ends. Internal training programmes for existing staff are valuable but slow, and the staff being trained still have day jobs to do.
None of these approaches systematically addresses the underlying problem, which is that the insurance sector does not have a reliable pipeline of junior data talent that is trained to the domain from the start. The consequence is that when a large insurer commits to building an AI-powered fraud detection capability, or a Lloyd’s syndicate decides to build a data-driven underwriting platform, the programme starts with a talent gap at the delivery layer that gets filled piecemeal and expensively.
Building the data team the programme actually needs
Our approach to Recruit, Train, Deploy addresses this directly. Rather than recruiting from the senior open market, Corecom Tech Academy recruits for aptitude and trains Associates specifically for the client’s data environment before they are deployed. For an insurer building a claims analytics platform, that means Associates arrive understanding the data structures they will be working with, the tools on the client’s stack, and the regulatory context that constrains how claims data can be used and modelled. For a Lloyd’s market firm building an AI underwriting capability, it means Associates who understand the Core Data Record standards, the placement data flows, and the specific Python and data engineering tooling the team works in.
Associates deploy on a two-year contract. No employment liability, no FTE commitment, no recruitment fee when the client converts them at the end. The domain knowledge builds inside the organisation over that period rather than walking out when a consultancy engagement ends.
The data function in insurance is becoming one of the most strategically important parts of the business. The firms that build it properly, with junior talent that genuinely understands the domain, will have a sustained competitive advantage over those still trying to hire their way out of the problem at the senior end of the market.
"What I find when I talk to heads of data in insurance is that the problem is not knowing what they want to build. It is finding people who can build it without spending the first year learning what insurance data actually looks like. That is what the training we do with our associates is designed to solve."
Rick Hughes, Director
Corecom Tech Academy
Looking to accelerate delivery across insurance teams?
Book a call today to discuss how we can strengthen your capability and reduce risk.
