In our latest wells, we’re not just counting barrels. We’re counting gigabytes.

Meet the team of data scientists working on an exciting new fibre-optic technology that’s processing big data from our wells. It’s not just the oil that’s valuable – the data is pure gold too.

The Fibra Frontier

Did you know Equinor has one of the fastest computing clusters in all of Europe? 40 servers process data from fibre optic cables in our wells and have already helped us save millions of NOK – and they’re only getting started.

Today, we’re boldly going where few have gone before us; into the depths below. As oil and gas finds its way from the reservoirs below Johan Sverdrup and onto shore at Kårstø, so does a vast amount of data.

“How?” you ask? By installing fibre optic cables along our wells. On top of the well sits an interrogator; a machine that sends laser pulses down the fibre, and when this laser bounces back it creates what’s called a back reflection. That’s when Fibra kicks in; helping us process these back reflections into useable data.

You might be asking yourself “how is that anything special?” and the answer is the sheer amount of data and the speed it’s gathered at. It's equivalent to constantly streaming 10,000 Netflix movies.

Fibra processes all these data at a speed of approximately 5 GB/s. This means that one of the biggest challenges they face is having to tune everything, and we mean everything, with speed in mind. While some software might have a couple of users performing actions here and there, Fibra has a computer that performs millions of actions each second.

“When we’re writing code, we need to keep in mind that each line will be running millions of times each second, so everything we do is optimised for performance. We run a ‘canary test’ every 30 minutes where we send a message through our system to make sure we get the right result on the other end,” Lindvar explains.

While we’re diving into the nitty, gritty details of Fibra today, it’s important to understand the rest of the process as well. After being processed by Fibra, data heads up into fo.tone – the software that visualises the data to our operators. See the video below for an intro, before we head further into the world of fibre optic cables!

From two weeks to two seconds

An important part in processing this data is creating a standard for streaming fibre data. Normally, each supplier has their own format for their respective interrogator, but Fibra has become “one system to rule them all” — or at least stream them all. This means our operators can view the data two seconds after it’s collected, an incredible speed increase compared to the previous way of working.

This means Equinor can rely on the data to support operations and maintenance – while they are performed. Two technologies sit at the heart of it; Kafka and Kubernetes. Kafka is used to save and receive data while Kubernetes is the platform that can process the data in near real time.

When you start an interrogation session, the first thing the system does is to choose the correct processing profile. Fibra always stores the raw data, but different algorithms are needed to process the various dataflows and look for artefacts within them. The selection is based on what kind of data you’re interested in.

“Different tasks have different needs, so if we’re doing a pressure test, we might only be interested in pressure related data, which the processing profiles let us filter for. This means we save a lot of resources, can more easily scale things and can make the most of our hardware,” Espen tells us.

Their code has to be efficient enough to process the data a little faster than the speed they are received at, if not it'll all just stop working.

“Fibra is continuously doing various signal transformations and looking at the specifics of the signal. That’s a standard way of doing things, but we’re running a number of transformations simultaneously and must optimise that flow in and out of the optimisation algorithms,” Espen explains.

A need for speed

Everything seems to be heading to the clouds these days, and the team did plan to do the same for Fibra. They quickly realised that the cloud simply wouldn’t be able to deliver the speeds they needed. As a result, Fibra does edge processing — with on-premises servers and hardware.

“We set up everything from hardware to Kafka and Kubernetes and handle the infrastructure in a way that lets us script everything with ease. It’s more than just developing code,” Ian Utbjoa says.

“There aren’t a lot of other projects that handle the amount of streaming data like we do. Moving it into the cloud wasn’t feasible from a technical or economic standpoint, because of the sheer speed we need to handle the data at,” Ian explains.

Working on Fibra means working with a need for speed, but it also means you’re being more than a developer.

Fibre optic cables and interrogators aren’t brand-new technologies; they have been used for decades to gather data. But not from wells offshore and not at this scale.

“We’re finding new ways to use relatively new technology, and it feels like a little like we’re innovators trying to find new, efficient ways to use the resources we have. That’s what makes working on Fibra so exciting,” Petter Kvamen Clausen says

Fibra is in use on Johan Sverdrup and Martin Linge but isn’t listed as business critical. That doesn’t mean they don’t act as if they are, however:

“Naturally, we want to be notified of any errors that might occur and we act like we’re business critical in everything we do. Monitoring is especially important for us, and setting up monitoring environments lets us keep a close eye on everything,” Petter explains.
They do this by using Prometheus and Grafana – open source monitoring software used all over the world.

“Not being business critical doesn’t mean that operators aren’t depending on the data being available, so we always work to deliver as stable a product as possible,” he adds.

Already saving millions

Fiber optic cables and Fibra have already proved their worth, and one example comes from Johan Sverdrup, where Fibra helped save us 40 million NOK.

As a geoscientist by background, Taber has now taken on the role as project leader for Fibra & Fotone, and is impressed by the work done by the developers.

“The fact that it was developed from scratch is a testament to their creativity and their perseverance. Developing a very stable and high-performing system that allows for scaling in just two years is just incredibly impressive,” Taber says.

Today, Fibra is in use on Johan Sverdrup and Martin Linge. In total, fibres in over 35 wells can be monitored in real time, with more to come soon. But how have they managed to achieve this kind of scaling and quick implementation?

It’s all thanks to a mindset of scripting and “automating themselves out of a job” — at least parts of it.

“Repeating manual processes isn’t just boring for us, it’s also a waste of time and brings little value. Since we’re a small team of five, our main principle is that if we can automate ourselves out of a job, we’re going to do it,” Espen says.

Opening more data

This automation and scripting focus has led them to being able to easily bring new wells into Fibra, but there might be other wells and assets heading their way soon too. While constantly improving and maintaining Fibra is one of their core tasks, one of the next big things is Fibra Light, Espen tells us:

“We’re building a smaller, mobile version of Fibra that we can bring to assets that want to use our real time monitoring system. This could be for intervention jobs, or assets that want to test Fibra to see if it is something they want to invest in,” Espen says.

Intervention jobs are performed on wells that require maintenance, repair or stimulation to improve or restore their productivity. These jobs are often done from a vessel,which means little chance of powerful internet cables to help transfer the data to land.

As a result, Fibra Light processes just the data they need to process on-site, and then sends the processed result to shore.

“Fibra Light aims to bring real time observation of well operations to all assets with very low effort and cost,” Lindvar says.

A large part of building Fibra Light is sheer logistics; developing a solution you can take along with you offshore and won’t take up too much space.

“For the initial phase and testing we’re sending people from our team offshore to run and test Fibra Light. The end goal is that “anyone” will be able to operate Fibra Light offshore, and that it can work with the different types of interrogators we have,” Espen explains.

The fibre optics project in Equinor has truly kicked off with full force and seems to be st(r)eaming ahead at full speed. Only time will tell how it all turns out, but this probably isn’t the last you’ve heard from the Fibra team – or from the fibre optics project in Equinor.

Further reading