This chart shows BHA system performance by year when the rotary steerable system is the primary drive mechanism. (Image courtesy of Schlumberger) 

For many years, the operator and service sectors have spent countless hours discussing the notion of good and poor drilling system performance. Almost all have attempted, with varying degrees of success, to measure this performance and to use that measurement to gauge themselves against their peers and competitors.

Success in such an undertaking comes down to ensuring that the metric used is really measuring what is important to your operation or activity. The critical question is, “What should the metric be?”

The answer is that the metric should truly measure what it is intended to measure. This may sound strange, but it is not uncommon for a key performance indicator (KPI) to measure only one of a multitude of facets that comprise performance. This approach usually ends in disappointment.

The metric should be defined so as to be easily understood and to ensure that data collected can be compared on an “apples-to-apples” basis without manipulation. It should be easily gathered and collated, and if possible, it should be available from two or more sources so that it can be cross checked for accuracy.

So can this single metric drive the desired performance improvement? It is common for more than one metric to be needed, but using more than two or three becomes far too complicated.

The process of choosing a suitable metric can be easily illustrated with an example from the drilling service provider sector (directional drilling, measurement-while-drilling, and logging-while-drilling). For many years, drilling service providers and their clients, the operators, have widely used mean time between failure (MTBF) to measure performance, drive performance improvement, and compare the relative merits of various providers.

Typically, operators want high system reliability and efficiency, and many are willing to reward providers over and above the contract rate to encourage performance. For the drilling service industry, reliability can be interpreted as placing the well in the correct place, providing high-quality measurements in both memory and real time, drilling efficiently (no trips caused by component or system failure), and finally ensuring that the hole drilled is fit for purpose to allow subsequent operations to progress without hindrance.

The question at this point is, “Does MTBF accomplish this?”

MTBF defines the reliability of a tool by dividing the total pumping hours the tool was exposed to by the number of failures during that time. A failure is normally defined as an event that causes lost time.

The first point to note here is that this approach only addresses the reliability component of an overall performance metric and assumes that if there are no failures, efficiency is achieved.

Secondly, MTBF is usually measured on a tool-by-tool basis, not in terms of system performance. It is assumed that if all the tools in the bottomhole assembly (BHA) have a high MTBF, the BHA has a high MTBF.

Lastly, MTBF uses pumping hours and assumes circulating time is the critical factor. If the tool can be circulated through longer without failure, performance is considered to be better.

It becomes clear that while MTBF is useful when looking at the life of components and erosion issues within tools, its use as a metric to drive drilling performance is limited. MTBF has little bearing, for example, on the footage drilled. There is an assumption that circulating time is an equivalent to the distance drilled, which certainly cannot be assumed.

Measuring the MTBF for a given tool does allow some basic reliability evaluations to be made when comparing one tool to another, (in terms of which tool functions for the longest pumping time), but this can be very misleading. It takes no account of the total BHA system in which that tool or service was run and whether all the BHA components used are compatible. It is total system reliability that is significant in this case, not the performance of a single component.

The use of pumping hours must also be questioned. While some broad comparisons can be drawn between circulating time and distance drilled, part of the evaluation is based on how efficiently the circulating time is used. It is a stretch to say that circulating off bottom or drilling slowly to reduce drilling stress actually makes the tool appear more reliable and by default, because of the gross association, efficiency has increased.

The metrics need to be improved to achieve the aim of measuring and driving improvements in drilling services performance.

Evaluating the approach

A number of issues should now be obvious: 

• One KPI cannot be used as a catch all. A small number of interrelated KPIs are more likely to capture the overall metric of performance.
• A tool cannot be considered in isolation, but must be evaluated as part of a system. The goal is to define a metric that reflects system performance.
• Maximizing system reliability is
critical, but is only part of the
challenge. How we use the time that reliability provides is equally important.

More factors than MTBF must be considered if a true evaluation is to be made. In fact, two or three KPIs are needed: one to address reliability, one to address efficiency, and if a tool that provides data is being evaluated, there must also be a KPI to measure the quality of the data and the amount of data obtained versus that required.

These KPIs must all be seen in terms of the overall goal ? that of drilling the well. It makes much more sense, in this case, to define the KPIs in terms of footage rather than time.

The following would be reasonable KPIs: 

• The reliability of the BHA system measured in feet between failures;
• The efficiency of the system in the number of feet drilled per circulating hour; and
• The amount of good quality data obtained at the required data density as a percentage of the total footage drilled.

These KPIs allow both operator and service provider to measure BHA reliability and efficiency as well as data quality/recovery. They also make it simple to compare how the complexity of the BHA affects these values. Finally, these KPIs allow a more objective comparison of drive mechanisms and vendors. While these KPIs are not perfect, they tie reliability, efficiency, and data recovery together in a simple way.

Showing the results

In this example, BHA reliability continues to improve, but drilling efficiency shows only minor improvements in terms of feet per circulating hour. By evaluating this issue using the KPIs listed above, additional work can be done to further define performance.

In this case, we would look at whether we are being too conservative with drilling parameters or choice of bit to avoid BHA failure. We shoud determine if the mud systems or rig is capacity constrained in terms of hole cleaning, and if penetration rates are being controlled for data collection. The distance being drilled per hour might be a result of a combination of these factors and many others, but now we are aware of where to look for performance improvement.

Going forward, the industry must look more closely at the system performance improvement we are trying to drive and what significantly effects performance. Appropriate KPIs reflect all of the major influences on final performance. To be useful, KPIs must also be easy to gather and cross reference to provide a true benchmark of current performance. Only then can performance goals be set realistically so that progress can be measured against them and providers can be benchmarked against one another.