Architectures and algorithms of an autonomous small-scale drilling agent

Journal Pre-proof Architectures and algorithms of an autonomous small-scale drilling agent Suranga C.H. Geekiyanage, Erik A. Loeken, Dan Sui PII:

S0920-4105(19)31253-7

DOI:

https://doi.org/10.1016/j.petrol.2019.106834

Reference:

PETROL 106834

To appear in:

Journal of Petroleum Science and Engineering

Received Date: 2 May 2019 Revised Date:

17 December 2019

Accepted Date: 19 December 2019

Please cite this article as: Geekiyanage, S.C.H., Loeken, E.A., Sui, D., Architectures and algorithms of an autonomous small-scale drilling agent, Journal of Petroleum Science and Engineering (2020), doi: https://doi.org/10.1016/j.petrol.2019.106834. This is a PDF file of an article that has undergone enhancements after acceptance, such as the addition of a cover page and metadata, and formatting for readability, but it is not yet the definitive version of record. This version will undergo additional copyediting, typesetting and review before it is published in its final form, but we are providing this version to give early visibility of the article. Please note that, during the production process, errors may be discovered which could affect the content, and all legal disclaimers that apply to the journal pertain. © 2019 Published by Elsevier B.V.

CRediT author statement  Suranga C.H. Geekiyanage: Conceptualization, Methodology, Software, Formal Analysis,  Investigation, Data Curation, Writing‐Original Draft, Writing‐Review & Editing, Visualization.    Erik. A. Loeken: Conceptualization, Methodology, Software, Investigation, Resources, Data Curation,   Writing‐Review & Editing, Project Administration.    Dan Sui: Conceptualization, Methodology, Validation, Resources, Writing‐Review & Editing,  Supervision, Project Administration, Funding Acquisition.   

DRAFT TITLE: ARCHITECTURES AND ALGORITHMS OF AN AUTONOMOUS SMALL-SCALE DRILLING AGENT. Author: Suranga C.H. Geekiyanage, Department of Energy and Petroleum Engineering, University of Stavanger, Norway Co-author: Erik A. Loeken 1 , Department of Energy and Petroleum Engineering, University of Stavanger, Norway Co-author and post-corresponding author: Dan Sui, Department of Energy and Petroleum Engineering, University of Stavanger, Norway ABSTRACT This paper describes the core architectures and algorithms of an autonomous small-scale drilling agent. The agent operates in a laboratory rig, demonstrating drilling scenarios with limited or even no human intervention. The work illustrates its performance through self-coordinating state transition, Rate of Penetration (ROP, drilling speed) optimization capability, formation classification, and drilling incidents management. The agent is an original rule-based system, and its control architecture utilizes finite states automation. The novel ROP optimization strategy employs a gradient search in Weight on Bit (WOB)-rotational speed (RPM, Revolutions per Minute) control parameter space. It generates an increasing ROP trend with time and requires re-iteration at abrupt formation changes. Several drilling incidents are managed using ‘if-then’ logic-based activity decomposition. A key learning outcome from the study is the comprehension of the requirement of standard software architecture and Applications Programming Interfaces (API) for continuous research and development of the agent. Such interfaces enhance interoperability between systems and stimulate innovative thinking among independent developers to produce a better-faster set of algorithms. Laboratory testing and evaluation is an essential part of promoting the adaptation of digital technologies for drilling automation. Such studies are a useful, safe, and cost-effective solution for testing, integrating and improving hardware, software, and data management before expensive full-scale testing and integration. KEYWORDS Drilling Systems Automation, Autonomous Agent, Applied Artificial Intelligence, ROP Optimization, Laboratory Rig. 1. INTRODUCTION With the rise of energy demand, exploration and drilling in remote and extreme environments have become desirable. Such working conditions possibly create high mental and physical stress to crews, leading to high turnover rates with limited experience workforce during operations. A study conducted by (de Wardt et al., 2013) shows a drastic amplification of accident rates associated with working hours after 10-12 hour shifts. Thus, human factors such as stress, experience, knowledge, and competency can result in inconsistent, inaccurate, and imprecise procedures, leading to NonProductive Time (NPT), risky operations and poor Health-Safety-Environment (HSE) statistics. Robotic drilling systems (autonomous/ semi-autonomous/ teleoperated), on the other hand, provide a game-changing solution to operate in extreme environments. It will reduce costs per well, create reliable, consistent, and HSE friendly drilling operations. For instance, robotic drilling solutions can operate 24/7-year-round and have shown an approximately 50% increase in tasks completed per hour compared to conventional methods (Watt et al., 2016). Moreover, robotic systems can be reprogrammed for upgrades and therefore create a higher versatility, interoperability (plug and play) 1

Present address and affiliation : Equinor ASA, Forusbeen 50, Stavanger, Norway E mail : [email protected] 1

and adaptivity potential towards the tasks that come along with rapid technological advancements (Soendervik, 2013). Therefore, intelligent drilling systems such as geo-steering down-hole equipment and microelectromechanical ground robots are likely to become a key feature in the next generations of drilling rigs (Hu and Qingyou, 2007). However, the drilling industry needs to prepare for a significant technological leap forward. This study presents a theoretical framework and a proof of concept of an autonomous drilling agent. The agent architectures and algorithms are developed and tested on a small-scale drilling rig that operates under laboratory conditions. Overall control architecture utilizes finite states (sequence) automation. In addition, artificial intelligence (AI) is applied for the ROP optimization and formation classification. The agent is also capable of identifying several undesired drilling incidents and mitigating such drilling problems in real-time. Main contributions are: 1. Proof of concept of a completely autonomous drilling agent. 2. A real-time ROP optimization algorithm and laboratory testing. Furthermore, the study includes an analysis of data pre-processing algorithm steps required for supervised formation classification. For this, authors utilize lab-data generated by the drilling agent, while drilling rock samples such as granite, cement and sandstone. Resulting data analysis indicates that physics-based feature construction is more suitable for formation evaluation than measurements such as WOB, motor torque, RPM, depth and ROP, especially when taken as paired input features for classification. Such comprehension is vital when developing and standardizing machine learning algorithms for rock recognition. Thus, this study primarily aims to advance the design of digital architectures and algorithms required for unmanned and remotely operated drilling facilities development, which are pivotal in offshore and extreme environment drilling operations. Such advancement has the potential to transform passive surveillance remote centers today into active decision support hubs in the future (de Wardt et al., 2015). 2. BACKGROUND Autonomous systems require sensors, actuators, and controllers. Auto-drilling processes require three elementary mechanical systems; circulation, rotation, hoisting and an electric power supply system. Sensors provide information regarding the status of these systems during real-time operations. Core measurements (i.e., drilling parameters) include WOB, motor torque, RPM, depth, pump pressure and flow rate. Process actuators (i.e., a mechanical element that converts energy into motion) are hoisting machinery (drawworks), top drive motor and mud pump(s). An agent (an ensemble of computer programs) performs decision-making via on-line data analysis, Figure 1.

Figure 1: Schematic diagram of a simple drilling agent. An agent is an ensemble software containing data management and decision-making algorithms. One can think of it as the ‘brain’ of the drilling rig. Inputs are received through sensors and output interactions with the 2

environment (i.e., drilling formation) occur through actuators. See also (Russell and Norvig, 2002) for the definition of such autonomous agents. (Macpherson et al., 2013) investigated the status of the drilling systems automation and identified the development of fully autonomous drilling systems (an agent) as a process requiring several tiers of technological advancements. They proposed three levels in this process. Tier 1. Advice a driller with optimal control parameters for decision-making, Tier 2. Semi-autonomous drilling systems, where a driller has the right to approve or reject, Tier 3. Fully autonomous drilling systems, where a computer agent decides and executes actions without the intervention from a driller. Ideally, an intelligent drilling agent in Tier 3 will plan, predict, diagnose, interpret, execute, control, communicate, and adjust its optimal operating parameters to improve drilling efficiency and reduce costs. An idea behind an intelligent drilling advisor is patented in (Edwards et al., 2012) and theoretical definitions of autonomous agents, in general, can be found in (Russell and Norvig, 2002). However, most drilling industry experts agree that completely independent systems without any human intervention are not yet achievable or practical (Parasuraman et al., 2000). Most research activities are still focused around Tier 1, and the drilling industry is gradually preparing for the big jump ahead (de Wardt et al., 2013). (de Wardt et al., 2015; Wilson, 2016a), through road maps, also proposed ways to accelerate drilling automation. They identified significant challenges and obstacles that the industry needs to foresee and overcome in order to mature drilling systems automation. Eight such challenges for potential elaborations are highlighted below. 1. The requirement of a systems architecture to describe hierarchically connecting elements. Systems architecture generates value from combining drilling devices and sub-systems. This promotes the interoperability of drilling systems (and sub-systems) and requires identification of workflows and interfaces. 2. Communication challenges such as connectivity, time delay or cybersecurity issues. They become critical due to industry-specific down-hole to surface communication challenges or in remote operations. 3. Instrumentation and measurement systems development for drilling automation and their availability, accuracy, precision, and redundancy requirements. 4. Drilling machinery, its role in drilling process mechanization (human effort replacement with machinery) and drilling efficiency. 5. Control systems development, testing and validation, for the standardization of drilling data flow schemes and integration of data-driven approaches with physics-based algorithms to address uncertainties during operations. 6. Models and simulations to describe drilling-process dynamics accurately enough to predict and control. Current status of such models, simulations and future goals are investigated in (Sugiura et al., 2015; Wilson, 2016b). Most of them currently run in an advisory mood, and the development of the better and faster collection of algorithms (for example, predictive, optimization, classification, and both prognostic and diagnostic anomaly detection algorithms) can move drillers into the supervisory level. 7. There is a gradual shift in the roles and tasks of rig personnel to automated machinery, algorithms, and remote centers. This transition requires ‘humans and systems integration’ in order to improve HSE, user-centered design practices, elegant interfaces, etc. 8. Certification and industry standards are as a ‘must’ in order to implement interchangeability of drilling components, lower installation plus maintenance costs, and increase design efficiency and safety. Many researchers are actively investigating how to overcome these challenges. (Ambrus et al., 2015) presented a methodology to overcome barriers in drilling automation. The (Norwegian Research 3

Center (NORCE), 2019) is currently investigating an approach to develop autonomous drilling systems in collaboration with several industrial sectors. Overcoming above challenges requires long-term plans in line with teams formed by experts from the oil and gas industry and the exploitation of state-of-the-art technologies from parallel industries such as aviation, space exploration or automobile industry, see (Godhavn and Hauge, 2018; Godhavn et al., 2011; Thorogood et al., 2010) for further information. Most importantly, many researchers agree that new solutions require multidisciplinary research and innovation. Therefore, a robust framework to integrate cross-disciplinary fields such as robotics, artificial intelligence, data science, cybernetics, mechatronics, and so forth with drilling engineering is anticipated. Drilling Systems Automation Technical Section (DSATS) of the Society of Petroleum Engineers (SPE), aims to tackle several of the above mentioned challenges such as development, testing and validation of software systems architecture, interoperability of machine components, development of instrumentation and measurement system and intelligent algorithms, by providing an interactive platform to promote autonomous drilling with the Drillbotics® annual student competition. See guidelines in (Drilling Systems Automation Technical Section (DSATS), 2018). The competition, which is a stepping-stone in the DSATS drilling automation roadmap, involves the development of laboratory-scale autonomous drilling rigs. Every year, the participants face new challenges, such as the ROP optimization, supervisory control, and directional drilling. One can find examples of such autonomous small-scale system designs in (Arnø et al., 2019; Loeken et al., 2018; Losoya et al., 2018). These rigs act as an infrastructure to establish multi-disciplinary studies necessary to overcome the challenges above. (Westermann et al., 2015) reviewed existing small-scale test rigs and utilized a mechanically downscaled rig to analyze drill string dynamics. (Cayeux et al., 2017) studied complications involving downscaling mechanics of full-scale drilling systems to the laboratory-scale. In addition, they validated that automated drilling systems are highly consistent when compared with manual operations. (Wiktorski et al., 2019) showed that such rigs are useful when it comes to sensor development and vibration studies by employing high-speed image acquisition. (Arnø et al., 2019) presented a state machine-based control architecture for autonomous drilling. They also mentioned a fit-for-purpose data acquisition-visualization system using a lab-scale rig. (Bilgesu et al., 2017) developed an AI algorithm for the drill-off test using a Drillbotics® competition rig. ROP optimization via minimizing Mechanical Specific Energy (MSE) approach was investigated by (Losoya et al., 2018). (Losoya, 2016) further examined vibrations management, remote control and surveillance through a smartphone application. Therefore, lab-Scale autonomous drilling systems can contribute towards developing, testing, and integrating sensors, data analytics and AI algorithms for drilling systems automation. In this study, the authors emphasize that in order to face the aforementioned challenges (particularly challenges 1, 5, 6 and 7), such small-scale drilling systems are useful to develop systems architectures, machine learning algorithms, data schemes and humans-systems integration. 3. THEORY 3.1. Mathematical Model of the Agent Architecture One can map an autonomous drilling process with defined boundary conditions (‘a stand’ in fullscale/lab-scale) as a sequential process arising due to linear or non-linear combinations of finite steps/states (McTiernan, 2016; Norwegian Research Center (NORCE), 2019). The execution order of the process sequences can vary upon drillers’ experience, drilling formations, weather, rig conditions, and so on. The automating process requires the explicit decomposition of tasks (states) and coordinating the state transition process. Process optimization involves minimizing the number of sequences and costs necessary to get from the initial state to the target. Such sequence modeling with finite states automation is a common technique applied in speech and natural language modeling (Coleman, 2005). 4

Define an agent,
as a quintuple.


≡ ,  , , , . 1

Then,    ,  , … … . ,   is a set containing ′′ number of finite core states that involve autonomous drilling activities. The starting state     ⊆ . That is, if S = {establish circulation, reach drilling conditions, drilling, evaluate remaining number of sequences to reach target depth, stop drilling, Faults Detection, Isolation, Recovery (FDIR), ROP optimization, etc.}, ‘Establish circulation’ or ‘evaluate the remaining number of sequences to reach target depth’ could be selected as  .  ⊆  is the target state and is the input set of cases for state transition. Each of the above core states can contain sub-layers of finite states to complete the task necessary to trigger the state transition ∶ ⨉ → . Thus,  ⊆  is developed upon rule-based and goal-based activity decomposition. Developed architecture (Figure 2) of autonomous drilling system is similar to the ‘subsumption’ architecture as known in the robotics and intelligence community (Brooks, 1986). Subsumption architecture has been utilized for intelligent agent design purposes in other domains (Connell, 1987). It arises from nesting (subsuming) one layer of finite states (sets) in another through a hierarchical manner. Key features of the architecture are: 1. Multi-goal oriented, 2. Multi-sensor based perception in all states, 3. Extensibility to evolve with time adding new control layers subsuming previous. Robustness of the performance depends on the controllers’ ability to transit between setpoints suggested by algorithms.

Figure 2: Finite states automation architecture of an autonomous drilling agent, depicting subsumed control (nested) layers. 3.2. Gradient Descent for ROP Optimization ROP is a multi-variable (continuous) function of parameters such as WOB, TOB (Torque-on-Bit, T ), RPM, flow rate, formation strength (generally represented as MSE), bit properties, and so forth. Denote these parameters as   WOB,   RPM, !  mudflow rate, and so on. Then "    ,  , ! , … . .  , and  ∈ $% . Thus, 5

&'(  "    ,

 , ! , … . .  . 2

The goal of the ROP optimization is to find an optimal set of coordinates " ∗ ≡  ∗ , ∗ , … ∗  such that " ∗   +, -+.+. Define a simple objective function /" for the ROP optimization with the Euclidean norm as 

+- /"  1" 2 &'(3456785 1 . 3 0

Then, for a constant ROP setpoint, minimizing /" is identical to minimizing the distance between " and the desired ROP setpoint. The gradient vector of the ROP at operating point " ≡ : , , , , !, , … . . , ; is B

@" @" @" , ,….., A <"=   >"   ? @  @  @ 

0C0D

. 4

FFFF= has the important property that at any point in the  2 1 search space, it ROP gradient <" always points to the maximal increase of the ROP. In other words, the gradient is always perpendicular to the ROP hyper-surface contour "  G, where G is an arbitrary constant, and G ∈ H. Once the search direction at any operating point is identified, multi-variable optimization reduces to a 1-D line search that requires moving along local gradients. Such a gradient descent method (Luenberger, 1984) is shown below (Figure 3): "=I%  "=I 2 JI <"=I . 5

Where J represents the step size (learning rate) and L is the iteration number.

Figure 3: Gradient descent illustrated in 2-D parameter space for the ROP optimization. Observe that J changes at each iteration and the proper selection of J can guarantee the convergence (Yuan, 2008) of the generated ROP sequence, or /"  M /"  M ⋯ M /"I . 6

Define a termination criteria ‖"I%  2 "I ‖ ≤ ∗ , where ∗ is the desired margin of termination error. Then the ROP gradient descent algorithm is: Set iteration counter L  0 Select initial coordinates of " ≡ : , , , , !, , … . . , ;. Do { 1. Compute the gradient to select direction, <:"=I ;. 2. Select the learning rate JI , where JI ∈ H% . 3. Update "=I%  "=I 2 JI <"=I  and L  L + 1.

While ‖"I%  2 "I ‖ M ∗ and " S8 ≤ "I ≤ "STU } " S8 ≤ "I ≤ "STU is the set of maximum and minimum constraints that confine the search space. However, ROP is a result of bit-rock interaction. Several theoretical and empirical ROP models can be found in the literature, see (Bataee et al., 2010; Teale, 1965). One such simple ROP model that uses the concept of MSE to represent formation properties are introduced in Eq. 6-7 (Edalatkhah et al., 2010; Hamrick, 2011). VW  or &'( 

X'Y.

Z[\85 4



+

2Z. &(V. ]\85 Z[\85  4 . &'(

2Z. &(V. ]\85

Z[  _ 4\85 ` VW 2 X'Y

, 6

. 7

Observe that drill bit and formation properties are not controllable. Therefore, it is required to assume that some of such parameters that affect ROP are relatively constant for a time. For instance, formation hardness (represented via MSE), hole-cleaning, temperature variations and cuttings transport effect on the ROP and bit diameter (D . Especially in laboratory-scale, the effect of the above parameters on ROP is relatively insignificant. Furthermore, significant changes in such parameters require the re-iteration of the algorithm to suggest optimal parameters for new conditions. 3.2.1. Instantaneous Gradient Computations for Real-time Implementation The instantaneous gradient of " with respect to " at any operating point ( L 5c iteration) can be estimated using linear regression. If "I  and "I are sampled at the same frequency for a time period d and e is the number of samples considered for obtaining instantaneous gradient at the iteration, >"I  

h h ef∑h 5C "I,5 :"I,5 ;i 2 ∑5C "I,5 ∑5C "I,5  



h e ∑h 5C:"I,5 ; 2 f∑5C "I,5 i

. 8

Since at least two samples are required to calculate a slope of the least square line, e ≥ 2. Observe that we can use Eq. 6 to estimate the gradient of measured depth [ with respect to time, i.e., instantaneous ROP [-m &'(]. Furthermore, methods such as ‘moving average’ can also be applied to calculate the -m &'(. For example, for a ∆d sampling time, indicator formula is

7

h

h%s

5C

5Cs

p[ 1 -m &'(   qr [5 2 r [5 t . 9 pd eV∆d

V is the window length of the moving average. 4. EXPERIMENTAL RIG 4.1. Mechanical Systems and Sensors

Figure 4 below shows a schematic diagram of the agent body. Tables 1 and 2 summarize the rig sensor information and important parameters and properties of the drill string assembly. Note that sampling frequencies for all sensors are synchronized ahead of each drilling operation. Sensors are acquiring the data from a single point and calibrated in reference to each other by a commercial (HBM Norway, 2018) data acquisition system, ensuring consistency and reliability of measurements.

Figure 4: Small-scale autonomous drilling rig (left) and a schematic representation of its’ mechanical systems and sensors (right). For information regarding the driller-machine interface, see Figure 17.

Table 1: Sensor information. Sensor W1, W2, W3

T1 T2 H ω P A V

Description Tri-axial load-cells: v , vertical component to measure hook load/surface WOB and U , w horizontal components to measure transverse forces affecting the drill string. Surface torque Drill pipe torque Vertical depth Surface RPM Pump pressure Acoustic sensor* for communication and bit-rock interaction wasted energy dissipation estimation High-speed camera as a vibration sensor*

(*Non-invasive, portable and under-development sensors).

8

Range 0--300

Units N

0--8.59 10-4 --103 0-100 0-3500 0-10 126--146

Nm MPa cm rpm Bar dB

0--530

frames/sec

Table 2: Key parameters and properties of the drill string assembly. Parameter Pipe material and length Pipe outer diameter Pipe inner diameter Bit material and outer diameter BHA material and length

Description and Units Aluminum 6061-T6 alloy, 914.4 mm 9.53 mm 7.75 mm PDC-213, 28.58 mm SS 316, 37.89 mm

4.1.1. Hoisting System The robotic hoisting system aims to provide sufficient WOB during drilling. It is capable of pressing down the drill string assembly to provide up to a maximum of 300N WOB. Three actuators are controlled in synchrony using a Proportional Integral Derivative (PID) controller (Ang et al., 2005). The feedback depth measuring mechanism of the hoisting system position (drawworks elevation) is conducted via two methods. 1. Step counting of stepper motors of the actuators; 2. Optical (laser) height measurement sensor mounted on the top plate of the rig to detect the elevation of the drill floor (which travels along with the drill string assembly on the laboratory setup). 4.1.2. Rotation System The central element in the power transmission chain of the rotary system consists of a hollow-shaft brushless motor, which transfers the torque directly to the drill string. The brushless motor allows the circulation of drilling fluid through its hollow shaft passage and is connected to the circulation system via the rotary union (and a developed flange on top). The motor is centered and mounted on top of the hoisting system plate. It is connected to the drill pipe via another flange set under the hoisting plate. Such flange also facilitates mounting of either a data swivel or additional sensors directly beneath the top drive. The remaining components of the power transmission chain of the rotation system include drill pipe, BHA and bit. A built-in closed-loop position, velocity, and current controller, which enable calculations of RPM and motor torque based on position feedback. 4.1.3. Circulation System The circulation system enables cuttings transport and lubrication of the drill bit. Selected drilling fluid is water, and the pump capacity is approximately 18.5 l/min at a maximum of 4.1 bara pressure. The pump provides an adequate minimum velocity margin of 0.5-0.7 m/s range (Cayeux et al., 2017) for cuttings transportation and hole-cleaning. The pump discharge pressure is monitored using a pressure transducer. As per lower margin, 3.2 bar pressure is selected for the pump and regular drilling operation is expected to commence around 3.7 bar pressure (11.5 l/min). 5. IMPLEMENTATION 5.1. System Design This section describes three intertwined architectures necessary for an autonomous drilling agent: hardware, control (intelligence) and data flow. These architectures are the result coupled with planning, decision-making and executing drilling-related tasks and processes of the rig. 5.1.1. Hardware Architecture

9

This architecture starts from sensors and ends at remote data storage, Figure 5. The hardware architecture is the basis of the other architectures and has three precise levels. These levels are defined to prioritize time-critical activities. The lowest level, i.e., Level I includes micro-controllers (Arduino Due) carrying out the most time-sensitive tasks for drilling, such as on-off or PID control of motors and actuators and data acquisition. Level II components interact with the driller during operations and perform the planning and optimization of drilling sequences. Level III indicates a remote server for post-analysis queries.

Figure 5: Layered hardware architecture based on action/reaction, planning and optimization, and storage. 5.1.2. Control Architecture The rig objects (i.e., actuators, pumps, motors, and so on) are defined in the digital domain using objects and classes data structure. For instance, actuators 1, 2, and 3 are objects in the single class ‘Hoisting’ and contains attributes such as direction, speed, name, and so forth. After that, the PID controller is introduced as a 'method' inside the hoisting class. All the hoisting objects can call the PID controller to execute simultaneously. Linking objects in different classes or interactions between objects are achieved through creating new attributes within the objects. During operations, different objects in different classes collectively perform a task to achieve a unique goal. This enables the definitions of states according to the task decomposition. Take the example of the drilling state (! , see Table 3); all hosting actuators synchronously move down, while trying to maintain a specific WOB using the PID controller. Simultaneously, the top drive and pumps operate at predefined operating setpoints. The set of such activities together defines the ‘normal drilling state’ of the small-scale rig. Once states are defined, the state transition is modeled. Programming of the state transition can be achieved with an array/linked list/hash table data structure, as depicted in Table 3. In addition, see Appendix, Figure A1. Table 3: State transition table   for the core layer of finite states. State (x

Descriptions

 -Initialize/Just started

Initializing the systems and data 10

y ∶ x⨉z → {x z| z} z~  

 -Calibration

! -Drilling € - Completed  - Abort and alarm ‚ - Resume drilling

.- Drilling fault/incident detected .- Remedial action for the incidents

acquisition (software initialization) This state includes sensors calibration, bit position identification and touch-bottom, establish circulation, and equipment calibration. Autonomous drilling with varying WOB and RPM setpoints suggested by the ROP optimization algorithm using PID control. Drilling is completed. An advisory state implemented to provide early warning on drilling risks and to prevent damages or unsafe operations. Drilling (normal state) commences after remedial action. Drilling incidents detection based on equipment or operating parameter threshold violations. Incident management via preprogrammed procedures, by changing operations to avoid damage.



!





€

.

€ 



-

!

.

-

-

.



.

‚

-

Activity decomposition for the laboratory-scale autonomous drilling system is in Figure 6, where     and  = {€ }. Inputs to the states can be represented as = {0, 1, 2}. The implemented core layer of finite states set is    ,  , ! , € ,  , ‚ , . , . . Input set is a unique set of cases to each state. For instance, for the calibration state of  , Case 0 would be the identification of the hoisting actuators position with respect to a given set of coordinates and will return to the same state as a result and move the drill bit to touch bottom, which initiates Case 1, and state transition happens to ! , to start drilling. Case 2 will be triggered if a component malfunction or a safety threshold violation is detected. Input 2 (Case 3) then will lead the state transition to  , where the driller gets informed or the operation gets aborted.

11

Figure 6: Implemented state transition diagram depicting subsumed finite state-layers of the architecture. The primary state transition is as follows. Once initialized by the driller ( ), the agent calibrates the hoisting system to bring the drill bit in contact with the formation (tag bottom). The goal fulfillment is observed via hook load increment. Feedback height sensing loops introduced in Section 4.1.1. are used to control the hoisting system. During  , it also evaluates the status of its circulation and rotation systems, and if the calibration is satisfactory, it transits into the independent drilling state (! ). Else, the agent can alert the driller (voice alert) or abort the mission (‚ and  ). The driller can access the drilling state (i.e.,! ) directly and provide standard optimal operating parameters, if necessary. Otherwise, completion of the calibration state takes the rig into autonomous drilling mode. Autonomous drilling involves real-time optimization of the ROP via sensor feedback from the drilling environment, generating a new augmented layer of finite states due to each control parameter combination. In addition, drilling incident detection (.) and management (. ) have subsumed layers of finite states to the core layer. 5.1.3. Data Architecture Data management is an essential component regarding planning, decision-making and execution infrastructure for the auto-drilling. We have programmed the agent in a similar data architecture to that of lambda (λ) architecture applied in big data streaming (Kiran et al., 2015). We divide the data flow originating from sensors into two: one branch for visualization (real-time streaming) and another for drilling optimization and formation detection (machine learning algorithms) in batch processing, Figure 7. Using this architecture, we not only perform pre-processing of drilling data before decision-making algorithms but also provide real-time visualization to the driller. Furthermore, it also ensures that data is not overwritten, system safety and data security. We store data for post analysis and information extraction. For further information regarding the data quality transformation methodology, see (Geekiyanage et al., 2018).

12

Figure 7: Batch data processing architecture of the autonomous rig, which is similar to the lambda architecture in big-data processing. 5.2. Algorithms 5.2.1. ROP Optimization (Dunlop et al., 2011) have developed and tested a similar ROP optimization algorithm in full-scale using PDC- bits. They argued that while on-bottom, only three parameters: WOB, RPM, and mudflow rate can be manipulated. They further suggested that although the flow rate significantly affects cuttings build-up, without a downhole mud motor, the direct effect on ROP from flow rate is minimal. This reduces the applied optimization to 2-D space of optimizing the WOB and RPM for higher ROP. A simplified version of the gradient descent is implemented for observability of the algorithm performance. The search direction is initially fixed parallel to one axis of WOB-RPM search space. Instantaneous ROP calculation is performed using Eq. 9 with a 60s time delay to allow drilling conditions to stabilize. See the detailed algorithm in Section 3. 5.2.1.1. Constrained Optimization In addition to physical constraints of operational parameters, setpoints suggested by the algorithm must not violate safety thresholds of the equipment to prevent damage to equipment and to reduce the risk of drilling incidents. An improved version of the algorithm can incorporate drill string dynamics into the ROP optimization, as a safeguard to make obtained WOB and RPM stay in the safe region, where drill string vibration is minimal to prevent damages due to the severe drill string vibrations, like stick-slip, whirl and bit bouncing, Figure 8.

13

Figure 8: An illustration of the ROP contours and constraints in the WOB, RPM state-space, adapted from (Dunlop et al., 2011). 5.2.2. Formation Classification Re-iteration of the ROP optimization algorithm at distinguishable formation changes is necessary due to its control parameter dependency. In essence, optimal WOB and RPM vary from one formation to another. Hence, a formation classification algorithm in parallel to the ROP optimization algorithm is implemented. Itis a quadratic Support Vector Machine (SVM) (Boser et al., 1992; Wang and Lin, 2014). An illustration of a linear SVM is in Figure 9. Such SVM assumes features are linearly separable in the feature space. Otherwise, one should cast features into a higher dimensional space where they can be linearly separable or use a non-linear decision surface (kernel) for separation.

14

Figure 9: An illustration of a linear Support Vector Machine (SVM) for a two-class classification problem in 2-D feature space. The aim of such SVM is to find a decision surface where margin is maximum. SVMs’ are popular since they have a robust mathematical derivation and they can prevent overfitting cleverly, see (Boser et al., 1992). In addition, they can handle a large number of features and hence, computationally attractive. A quadratic SVM is an SVM where the decision surface is a paraboloid function (Sammut et al., 2010). Input features to SVM are constituents of surface WOB measurement, height, RPM and motor torque. Data were collected by drilling several formations such as cement, granite and sandstone at 9600 Hz sampling rate. Then time-series data were segmented, aggregated and labeled into a single data frame. After that, data pre-processing and scaling were performed. Then statistical features such as mean, median, kurtosis, etc. from the timedomain data were constructed. Besides, some frequency-domain features, for example, highest amplitude frequencies were extracted. The complete dataset contained 13,397,760 samples. 70% of the data were utilized for training and 30% for cross-validation. Further details regarding the algorithm development are presented in (Nilsen and Hjelm, 2018). Several others have investigated the identification of the rock type at the bit using similar data-driven approaches; see (Klyuchnikov et al., 2019; Mostofi et al., 2011; Nilsen, 2015; Soroush et al., 2010). Implementation of such supervised algorithm for formations classification has become standard and streamlined due to python libraries such as scikit-learn (Pedregosa et al., 2011). However, model performance accuracy strictly depends on the data pipeline designed for the machine learning model (Kotsiantis and Kanellopoulos, 2006). Therefore, crucial steps (Figure 10) are identified in the supervised formation classification algorithm and reasoning next.

15

Figure 10: Supervised formation classification algorithm workflow. All inputs to a supervised numerical classifier need to be in one frequency, and they cannot handle missing (NaN) values. Since different sensors sample in different rates and locations, a solution can be missing value removal or interpolation/extrapolation/imputation. Sensor data aggregation primarily addresses this challenge. Machine learning models can be skewed if outliers/out of range data are present (Hodge and Austin, 2004). Hence, a data-cleansing step is necessary. The presence of outliers can be identified via Tukey Inter Quartile Range method (Kotsiantis and Kanellopoulos, 2006). ‘Signal to Noise Ratio’ check, as defined by (Nadarajah and Kotz, 2007), can further indicate the requirement of outlier filtering. Filters such as moving average or median can also remove outliers. Filters during data cleansing steps can introduce time delays, so one might need to revisit the data aggregation step for input frequency adjustments. The raw measurements could appear noisy due to drill string dynamics, and removal of noise/outliers can remove early symptoms of drilling anomalies/process outliers such as kick and loss of circulation. Sensor measurement ranges are different and large numbers/ranges can create a bias compared to smaller ones if they are not scaled-down to one range during classification. According to (Hastie et al., 2009), normalization and standardization are tools for feature scaling. However, the scaling method should complement the model objective function, and not every model requires input scaling. Examples of such models are linear regression and decision trees. The minimum, the maximum, and the mean value required for scaling should come from the complete dataset (training and test datasets) during post-analysis. Supervised classifiers can use statistical and artificial features in addition to measurement variables, for example, mean or a quantile (P25/P75) of a time series. Drilling physics/mechanics based engineered features such as ROP, ROP gradient, MSE or Depth of Cut (DOC, bit displacement per revolution) for formation classification can reduce model error (Klyuchnikov et al., 2019). Therefore, artificial feature construction is an essential step in model performance Improvement. However, not all the features carry equal weight for statistical classification and some can add noise to reduce the model performance or have no cause/correlation to the classification problem. Feature redundancy can lead to model overfitting. Hence, a thorough post data analysis for feature selection is a prerequisite for identifying the best set of features for classification. Automated feature selection is also possible (Guyon and Elisseeff, 2003). Extraction of features during real-time can be challenging, especially if time-series data for statistical feature extraction/big-data volumes are considered for formation classification. Again, this step can introduce latency effects. If a weak model such as fuzzy-logic or logistic regression is selected for rock recognition, the ensemble technique boosting (Schapire, 2003) has proven to be useful (Kadkhodaie-Ilkhchi et al., 2010). Cross-validation is applied to understand the quality of the trained model predictions when presented with new data. In the case of poor prediction accuracy, one can reconsider primarily feature selection and previous data cleansing steps to improve accuracy. Another data 16

reduction/transformation technique, such as ‘feature extraction’, can be a part of this algorithm as well. However, it is subjected to the dimensionality/volume of the data and computational power available. Principle Component Analysis (PCA) is another method of dimensionality reduction (Wold et al., 1987). Data transformation into logarithmic scale can highlight data skewness and support classification, as illustrated in (Geekiyanage et al., 2019a). A supervised machine learning model for formation classification should, therefore, concisely address all the above crucial steps in data preprocessing before deployment. 5.2.3. Drilling Incidents Detection and Recovery The drilling agent is also capable of detecting and mitigating several drilling incidents such as twistoff, over-torque, stick-slip, damaging axial and torsional vibrations, stuck pipe, leak, and so on. For example, if the rig sensors detect continuous oscillations of the top drive torque beyond identified and hardcoded thresholds, damaging torsional vibrations is suspected, and a list of pre-programmed remedial actions (such as reduction of RPM) are iterated before re-initialization of the auto-drilling process. Under manual operating configurations, warning alarms for these costly incidents are provided to the driller using HMI and voice alerts. An overview of the implemented drilling incident detection methods and their immediate remedial actions are listed in Table 4. In addition, two of the drilling incident remedial algorithms developed and implemented are in Figures 11-12. Boundary constraints for such algorithms are introduced as: for example, for pack off; if pump pressure > predefined threshold, and then execute remedial action. Several drilling incidents can co-occur, for instance, axial and lateral vibrations in a coupled mode. Therefore, during programming, drilling incidents are ranked according to their severity and priority of the remedial actions are selected accordingly. Such ranking of incident severity can be diverse according to the drillers’ experience, rig design, and drilling formation properties, etc. Furthermore, if the first remedial action fails to restore the desired operating conditions after a time, a second remedial action (i.e., an alternative ‘then’ condition), a warning or a shutdown is programmed.

Figure 11: Pack off remedial action algorithm.

Figure 12: Over-pull remedial action algorithm.

17

Note that parameters x and y should be pre-determined by the driller considering variables such as rig design, drill string and formation properties, etc. Table 4: A summary of the drilling incidents that can be detected by the agent, their detection symptoms and preprogrammed first remedial action. Note that margins of RPM and WOB changes shown under the first remedial action column are tailored for the small-scale rig. Drilling Incident

Main symptom(s) used for detection (If)

Axial vibrations

Fluctuations of the surface WOB, i.e., load cell v measurement ocillating over predefined safety threshold and a drop in the overall ROP (estimated every 60 s).

Implemented first remedial action(s) (Then) Increase WOB by a predefined margin, for example, 20% and decrease RPM by the same predefined margin.

Load cell U and w measurement fluctuations over predefined safety margin, an increase in torque, and drop in the overall ROP.

Increase WOB by a predefined margin, for example, 10% and decrease RPM by the same predefined margin.

Transverse vibrations

Torsional vibrations/Stick slip

Motor torque and RPM periodical oscillations

Increase RPM by a predefined margin, for example, 10% and decrease WOB 5%.

Over torque

Motor torque over its safe operating window. Motor torque over its maximum threshold, an increase in WOB and overall ROP decreased. Over pull is only considered when tripping out of the well, or reducing the WOB. If the vertical tension, v in the load cells should increase above the hook load that was estimated during the system calibration, an over pull is detected,. A drop in pump pressure below its minimum threshold. Pressure loss scenario is simulated by opening the leak valve, see Figure 4. An increase in the pump pressure, motor torque (optional condition) and a drop in the overall ROP. Over pressure scenario is simulated by closing the 3-way valve, see Figure 4.

Decrease RPM by a predefined margin. Decrease RPM by a predefined margin.

Twist off

Stuck pipe/ Over pull

Leak

Over pressure/ Pack off

Stop upward hoisting movement immediately and try to tag bottom, see Figure 12.

Re calibrate the circulation system and if condition unchanged then alarm the driller. Pull to maximum allowable over pull value (determined by dividing the yield strength of the drill pipe with a safety factor of 1.1.) and attempt to gain circulation, see Figure 11.

6. RESULTS AND DISCUSSIONS 6.1. Performance Evaluation The agents’ state transition, while drilling a uniform formation, is in Figure 13. Incremental RPM changes are observed due to ROP optimization algorithm at work. However, WOB step changes are 18

not apparent due to the drill string axial vibrations, and a robust WOB controller is required for setpoint tracking. Severe WOB oscillations reflect the drill string axial vibrations. Further, the instantaneous ROP estimated by Eq. 9 relies on the accuracy of measured depth, which is a surface measurement. In full-scale operations, drill string dynamics, hole cleaning, cuttings transport, inclination, geo-temperature effects etc. will have a more significant impact on the ROP than in the laboratory scale setup, see (Busahmin et al., 2017). Therefore, surface ROP is different from the true downhole ROP. By incorporating the drill string dynamics models to estimate and predict the downhole conditions based on surface measurements is an ongoing challenge in drilling automation (Downton, 2012). Studies that combine the drill string dynamics (and other un-controllable parameters that affect the downhole conditions) to predict and control the ROP are found in the literature. For instance, a robust method to estimate the ROP using a non-linear drill string dynamic model was presented by (Ritto et al., 2010). Moving horizon estimator to predict the ROP that includes hole cleaning and wellbore pressure margin constraints was developed by (Sui and Aadnøy, 2016). A similar study conducted by (Soukup et al., 2017) in full-scale has demonstrated that direct drill string dynamics measurements at the drawworks can enhance their auto driller performance. (Bavadiya et al., 2017) examined a fully automated drilling rig to study the effects of drilling parameters on drill string vibrations and ROP. One of their conclusions was that axial vibrations increased when RPM was near to the natural frequency of the drill string, which increased ROP. (Hegde et al., 2018) employed a modeling method that combines data-driven and physics-based models to predict and optimize the ROP and concluded that hybrid approach provides higher accuracy (than deterministic models) and higher interpretability than an entirely data-driven approach. However, integrating drill string dynamics and other such constraints with the developed ROP optimization strategy in this work requires more investigation and adding downhole sensors and wired drill pipe for data transmission. Therefore, it would be better to consider the drill string dynamics effect (as non-linear constraints) in the ROP optimization in a future study. Learning rate selection (step size) requires careful attention as well, and there is an upper bound for J (learning rate) that depends on the gradient, see (Gupta et al., 2019). Otherwise, unstable operations can occur due to the heavy fluctuations of WOB and RPM. Furthermore, WOB and RPM are not orthogonal to each other, and there exists a physical constraint between them (Detournay et al., 2008). Nonetheless, an increase in ROP is obtained due to the ROP optimization algorithm.

19

Figure 13: The primary state transition process in progress according to the implementation methodology introduced under Section 5.1.2. , refer Figure 6 and Table 3. i.e., from Initialize/startCalibrate-Drilling-Complete, during autonomous drilling of a uniform (cement) formation. Notice the evident RPM variations due to the setpoints suggested by ROP optimization algorithm during typical ‘Drilling State’, increase of ROP and its 60s delay due to computations. ROP is calculated using the hoisting actuator step counting (elevation sensing) mechanism introduced under Section 4.1.1. Figure 14 shows the agents’ capability of managing a drilling incident. Observe the torque oscillations and motor torque approaching its maximum constraint. Then, the agent lowers the RPM to mitigate over-torque scenario and transits to drilling incident(s) detection and recovery augmented finite states (management) layer. The driller gets a warning at this stage, and afterward, the agent aborts the operation due to equipment safety and well integrity. Several drilling incidents such as loss of circulation, pump over-pressure, stick-slip, etc. can be simulated using the agent as well. Drilling incidents data are challenging to obtain (or to correctly identify or isolate), and the rig provides a valuable opportunity in simulating several drilling incidents in the laboratory.

20

Figure 14: Drilling incidents management capability (Motor torque oscillations and over-torque prevention) during autonomous drilling. Observe the state transition from Start-Calibrate-DrillingDrilling Incidents Detection - Aborted, according to the state transition methodology introduced under Section 5.1.2. , refer to Figure 6 and Table 3. MSE calculation utilizes Eq. 6. Implemented formation classification algorithm can work together with the ROP optimization agent. We are currently re-investigating data pre-processing techniques, and feature selection, see (Loeken and Loekkevik, 2019). A data visualization chart prepared for feature selection during classification algorithm development to identify granite (red) from sandstone (blue) is in Figure 15.

21

Figure 15: Scaled data (i.e., no units of measure) visualization chart prepared for feature selection step during classification algorithm development to separate granite (red) and sandstone (blue), see the complete algorithm workflow for data preparation steps in Figure 10. Notice the overlap of data points, making liner classification unrealistic with the selected features. Data distribution histograms of five variables (diagonal sub-plots) show that these features are incapable of separating granite from sandstone, suggesting additional features should be engineered. The remaining sub-plots shows the bi-variate feature distributions of the two formations. According to Figure 15, it is not possible to classify/separate these two using a linear classifier without casting data into a higher dimension with a PCA/feature extraction. Otherwise, a quadratic classifier function can be considered.

Figure 16: Variation of MSE, (i.e., energy input per unit rock volume removed) and ROP, when transitioning from a hard formation (T, tiles) to a softer (C, cement). MSE calculation utilizes Eq. 6, and ROP is estimated using the hoisting actuator step counting (elevation sensing) mechanism introduced under Section 4.1.1.

22

Monitoring ROP and MSE, however, has shown to be sufficient to detect (abrupt) formation hardness changes without supervised learning, see Figure 16 and notice the gradual ROP and MSE changes when transitioning from a hard formation to a softer one. This indicates that physics-based feature construction (such as ROP or MSE) is more suitable for formation classification than individual drilling parameters such as RPM, WOB, surface motor torque and depth. The classifier development process in the laboratory has provided a valuable understanding of the key steps required for a rock recognition algorithm with machine learning. 6.2 Discussions The agent can also operate assisting the driller (in manual mode) in addition to the fully autonomous mode. In the manual mode, the driller selects inputs such as RPM, WOB, and so on. Then the system executes drilling according to these parameters. New mechanical components with built-in sensor systems such as a BHA with accelerometers (vibrations management), gyroscope (well inclination) and a strain gauge (torque-on-bit), are in an early prototype stage and are currently being further developed to facilitate smart drilling applications. The developed HMI (Figure 17) has the capability of assisting drillers with real-time information such as operating parameters (surface WOB, RPM and pump pressure), ROP, vibrations levels, MSE, and so forth.

Figure 17: A part of the Human-Machine-Interface, depicting some of agents’ drilling incident management capabilities, depth-based well log and real-time information such as operating parameters, ROP, MSE and so forth. Note that downhole information shown here, such as bit RPM, downhole vibration levels, etc., are currently being developed as a part of the ongoing project. Also, see Future Work section and (Geekiyanage et al., 2019b). Small-scale robotic and autonomous drilling systems have a wide range of applications, see (Camarillo et al., 2004; Devlieg and Szallay, 2010; Kihlman et al., 2010). Their development is useful for agents drilling small holes in robust structures for screws (Bi and Liang, 2011; Rosenlund, 2017), and surgical drilling at medicinal settings (Lee et al., 2004; Moustris et al., 2011). They can be valuable for drilling on the lunar surface or asteroid belt for sample collection (Zacny et al., 2011), besides to excavate oil, gas or minerals in extreme terrestrial environments (Bar-Cohen and Zacny, 2009). Lab-scale autonomous drilling helps us to focus on understanding, evaluating, standardizing and investigating the technologies applicable in drilling automation. However, lab-scale drilling has its limitations. One limitation of the drilling machine is that it is not scaled down to mimic the complex dynamics of drilling systems. Instead, it is developed as a data-driven or reduced-physics testing environment to facilitate drilling systems automation. Therefore, it creates a cost-effective infrastructure for testing machine learning models and AI algorithms. Moreover, it can simulate drilling anomaly related data sets, develop drilling data analytics, and standardize interpretations. 23

Learning outcomes from the development process include the identification of a standardized software architecture for continuous research and development work. Standardized software architecture can provide an interface for algorithms developed by independent developers. API development and testing is an essential step in interoperability improvements necessary to meet the challenges of drilling automation. Moreover, we notice that the autonomous drilling rig has excellent potential to strengthen the cooperation with the public (research) and private sectors. 7. CONCLUSIONS This study presents a proof of concept of core architectures and algorithms of an autonomous drilling agent. The agent body is a lab-scale robotic rig. It can either assist the driller or drill autonomously, with increasing ROP, while identifying significant formation changes and mitigating several drilling incidents. One can compose an autonomous drilling agent through a sequence modeling-based control architecture. In addition, the agent requires a complementary hardware design and a data management system. The control methodology involves finite states automation. Autonomous drilling activities will compose the (horizontal) core-layer of finite states in the control architecture. State transition can be rule-based. Augmented layers of finite states such as drilling incidents detection, isolation and recovery can be subsumed to extend the architecture vertically. ROP optimization is programmable via gradient descent method. A supervised classification algorithm can perform rock recognition. However, monitoring ROP and formation strength changes are sufficient to indicate the formation change detection necessary to assist ROP optimization. Drilling incidents management follows rule-based activity decomposition. Sequence modeling for finite states automation and ROP optimization with non-linear programming shows promising potential for applications in drilling automation and optimization. The combined effect of architectures and algorithms create the illusion of primary drilling intelligence required for auto/semiauto/teleoperated small- scale operations. Laboratory-scale drilling (rig) provides a useful test-bench for developing, testing, integrating and troubleshooting digital drilling solutions before expensive full-scale implementation. It also facilitates the development of AI algorithms, data analysis tools, interpretation methods, interoperability, and Humans and Systems Integration. Autonomous drilling in the laboratory brings multi-disciplinary research necessary to overcome drilling systems automation challenges and promotes the integration of state-of-the-art technologies. 8. FUTURE WORK Future work involves software architecture standardization, API development, novel sensor testing (high-speed camera and acoustic), speech activation of the agent to communicate with the driller and tuning machine learning model parameters for better performance. Furthermore, monitoring and control of down-hole temperature effects on drilling rate optimization are suggested as a part of future work. The project also investigates directional drilling, scaling-up of the mechanical systems to replicate better drilling dynamics, and scaling of algorithms. ACKNOWLEDGMENTS Thanks go to NORCE for their support in helping to develop the rig and the agent. We also thank all the past and present members of the UiS Drillbotics team, Prof. Bernt Aadnøy, Dr. Tomasz Wiktorski and Ekaterina Wiktorski at the University of Stavanger. We acknowledge the Equinor academia program and Aker BP for funding the ongoing project. NOMENCLATURE

24

]\85 [\85 , ƒ, z, " " <", >" /" G " "∗ n N k J [ V ∆d t -m &'( W1, W2, W3 T1 T2 H ω P A A V 8

  λ U , w , v

Torque on Bit, Nm Bit Diameter, mm Variable (or a feature) Function of the variable " Gradient vector of the " Objective function An arbitrary constant, a real number Initial value of " Optimal value of " A positive integer Number of samples Iteration number Learning rate (step size) Measured depth, cm Window size Sampling time, s Time, s Instantaneous ROP, cm/s Surface WOB load-cell measurements, N Surface torque of the top drive motor, Nm Drill pipe torque, MPa Vertical depth between the rock and the drill floor, cm Surface RPM, rad/s Pump pressure, bar Acoustic sensor, dB Agent High-speed camera (vibration) sensor, frames/s ith state of the drilling rig auto-operation sequence Input trigger (vector) for the state transition State transition method (table) Final State Initial State Lambda (data) architecture Load-cell strain gauge force measurements on x,y and z directions respectively.

ABBREVATIONS AI API BHA DOC DSATS HMI HMI HSE MSE MWD NaN NPT P25, P75 PCA

Artificial Intelligence Applications Programming Interface Bottom hole Assembly Depth of Cut, vertical distance travelled per one revolution of the bit, cm/rad/s Drilling Systems Automation Technical Section Human-Machine-Interface Human-Machine-Interface Health-Safety-Environment Mechanical Specific Energy, MPa Measurement While Drilling Not a Number Non Productive Time 25th Percentile, 75th Percentile Principal Component Analysis 25

PDC PID ROP RPM SPE SS SVM TOB WOB

Polycrystalline Diamond Compact (bits) Proportional Integral Derivative (Controller) Rate of Penetration (Drilling Speed), cm/s Revolutions per Minute, rad/s Society of Petroleum Engineers Stainless Steel Support Vector Machine Torque on Bit, Nm Weight on Bit, N

26

REFERENCES Ambrus, A., Pournazari, P., Ashok, P., Shor, R., van Oort, E., 2015. Overcoming Barriers to Adoption of Drilling Automation: Moving Towards Automated Well Manufacturing. https://doi.org/10.2118/173164-ms Ang, K.H., Chong, G., Li, Y., 2005. PID control system analysis, design, and technology. IEEE Trans. Control Syst. Technol. https://doi.org/10.1109/TCST.2005.847331 Arnø, M., Thuve, A., Knoop, S., Hovda, S., Pavlov, A., Florence, F., 2019. Design and Implementation of a Miniature Autonomous Drilling Rig for Drillbotics 2018. https://doi.org/10.2118/194226-ms Bar-Cohen, Y., Zacny, K., 2009. Drilling in Extreme Environments: Penetration and Sampling on Earth and other Planets, Drilling in Extreme Environments: Penetration and Sampling on Earth and other Planets. https://doi.org/10.1002/9783527626625 Bataee, M., Kamyab, M., Ashena, R., others, 2010. Investigation of various ROP models and optimization of drilling parameters for PDC and roller-cone bits in shadegan oil field, in: International Oil and Gas Conference and Exhibition. SPE, Beijing, China, p. 10. https://doi.org/10.2118/130932-MS Bavadiya, V.A., Alsaihati, Z., Ahmed, R., Gustafson, K., 2017. Experimental investigation of the effects of rotational speed and weight on bit on drillstring vibrations, torque and rate of penetration, in: SPE Abu Dhabi International Petroleum Exhibition and Conference 2017. SPE, Abu Dhabi, UAE, p. 22. https://doi.org/10.2118/188427-MS Bi, S., Liang, J., 2011. Robotic drilling system for titanium structures. Int. J. Adv. Manuf. Technol. https://doi.org/10.1007/s00170-010-2962-2 Bilgesu, H., Cox, Z.D., Elshehabi, T.A., Smith, C., Idowu, G.O., Florence, F.R., 2017. A Real-Time Interactive Drill-Off Test Utilizing Artificial Intelligence Algorithm for DSATS Drilling Automation University Competition. https://doi.org/10.2118/185730-ms Boser, B.E., Guyon, I.M., Vapnik., V.N., 1992. A training algorithm for optimal margin classifiers, in: Proceedings of the Fifth Annual Workshop on Computational Learning Theory. ACM, pp. 144– 152. https://doi.org/10.1.1.103.1189 Brooks, R.A., 1986. A Robust Layered Control System For A Mobile Robot. IEEE J. Robot. Autom. https://doi.org/10.1109/JRA.1986.1087032 Busahmin, B., Saeid, N.H., Alusta, G., Zahran, E.S.M.M., 2017. Review on hole cleaning for horizontal wells. ARPN J. Eng. Appl. Sci. 12, 4697–4708. Camarillo, D.B., Krummel, T.M., Salisbury, J.K., 2004. Robotic technology in surgery: Past, present, and future. Am. J. Surg. https://doi.org/10.1016/j.amjsurg.2004.08.025 Cayeux, E., Sui, D., Akisanmi, O., Alani, O., 2017. Challenges in the Automation of a Laboratory-Scale Drilling Rig and Comparison with the Requirements for Full Scale Drilling Automation. https://doi.org/10.2118/185898-ms Coleman, J.S., 2005. Introducing speech and language processing. Cambridge university press. Connell, J., 1987. Creature Design with the Subsumption Architecture., in: IJCAI. pp. 1124–1126. de Wardt, J.P., Behounek, M., Chapman, C., Putra, D., 2013. Drilling Systems Automation - Preparing for the Big Jump Forward. https://doi.org/10.2118/163422-ms de Wardt, J.P., Macpherson, J.D., Zamora, M., Dow, B., Hbaieb, S., A;, M.R., Laing, M.L., DiFiore, A.M., 27

Inabinett, C.E., Anderson, M.W., 2015. Drilling Systems Automation Roadmap - The Means to Accelerate Adoption, in: SPE/IADC Drilling Conference and Exhibition. Society of Petroleum Engineers (SPE), London, England, UK, p. 19. https://doi.org/10.2118/173010-MS Detournay, E., Richard, T., Shepherd, M., 2008. Drilling response of drag bits: Theory and experiment. Int. J. Rock Mech. Min. Sci. https://doi.org/10.1016/j.ijrmms.2008.01.010 Devlieg, R., Szallay, T., 2010. Applied Accurate Robotic Drilling for Aircraft Fuselage. SAE Int. J. Aerosp. https://doi.org/10.4271/2010-01-1836 Downton, G.C., 2012. Challenges of modeling drilling systems for the purposes of automation and control. IFAC Proc. Vol. 45, 201–210. https://doi.org/10.3182/20120531-2-NO-4020.00054 Drilling Systems Automation Technical Section (DSATS), 2018. Drillbotics® [WWW Document]. Soc. Pet. Eng. URL https://drillbotics.com/about-drillbotics/ (accessed 6.23.19). Dunlop, J., Isangulov, R., Aldred, W.D., Sanchez, H.A., Flores, J.L.S., Herdoiza, J.A., Belaskie, J., Luppens, C., 2011. Increased Rate of Penetration Through Automation. https://doi.org/10.2118/139897-ms Edalatkhah, S., Rasoul, R., Hashemi, A., 2010. Bit selection optimization using artificial intelligence systems. Pet. Sci. Technol. 28, 1946–1956. https://doi.org/10.1080/10916460903160818 Edwards, M.L., Gongora, C.A., Iyer, G., McGriffy, D., McNeill, J., Shafer, D.F., Tolleson, C., Wight, D., 2012. Intelligent drilling advisor. US8121971B2. Geekiyanage, S.C.H., Ambrus, A., Sui, D., 2019a. Feature selection for kick detection with machine learning using laboratory data, in: 38th International Conference on Ocean, Offshore and Arctic Engineering (OMAE). ASME, Glasgow, p. 13. https://doi.org/10.1115/OMAE2019-95496 Geekiyanage, S.C.H., Løken, E.A., Sui, D., Wiktorski, T., 2019b. Architectures and Algorithms for a Smart Drilling Robot, in: 38th International Conference on Ocean, Offshore and Arctic Engineering (OMAE). ASME, Glasgow, p. 10. https://doi.org/10.1115/OMAE2019-95486 Geekiyanage, S.C.H., Sui, D., Aadnoy, B.S., 2018. Drilling data quality management: Case study with a laboratory scale drilling rig, in: Proceedings of the International Conference on Offshore Mechanics and Arctic Engineering (OMAE). ASME. https://doi.org/10.1115/OMAE2018-77510 Godhavn, J.-M., Hauge, E., 2018. Drilling Automation-Some Industrial challenges and Solutions, in: 4th International Colloquium on Nonlinear Dynamics and Control of Deep Drilling Systems. Stavanger, pp. 109–120. Godhavn, J.M., Pavlov, A., Kaasa, G.O., Rolland, N.L., 2011. Drilling seeking automatic control solutions, in: IFAC Proceedings Volumes (IFAC-PapersOnline). https://doi.org/10.3182/20110828-6-IT-1002.00551 Gupta, C., Balakrishnan, S., Ramdas, A., 2019. Path Length Bounds for Gradient Descent and Flow. arXiv 52. Guyon, I., Elisseeff, A., 2003. An Introduction to Variable and Feature Selection André Elisseeff. J. Mach. Learn. Res. 1157–1182. Hamrick, T., 2011. Optimization of operating parameters for minimum mechanical specific energy in drilling. https://doi.org/10.2172/1060223 Hastie, T., Tibshirani, R., Friedman, J., 2009. Springer Series in Statistics The Elements of. Math. Intell. https://doi.org/10.1007/b94608 HBM Norway, 2018. Data Acquisition Systems & Instruments [WWW Document]. URL 28

https://www.hbm.com/en/5502/daq-data-acquisition-systems/ (accessed 6.23.19). Hegde, C., Soares, C., Gray, K., 2018. Rate of penetration (ROP) modeling using hybrid models: Deterministic and machine learning, in: Unconventional Resources Technology Conference. SPE/AAPG/SEG Unconventional Resources Technology, Houston, Texas, p. 19. https://doi.org/10.15530/URTEC-2018-2896522 Hodge, V.J., Austin, J., 2004. A survey of outlier detection methodologies. Artif. Intell. Rev. https://doi.org/10.1023/B:AIRE.0000045502.10941.a9 Hu, Q., Qingyou, L., 2007. Intelligent Drilling: a Prospective Technology of Tomorrow, in: International Oil & Gas Conference and Exhibition. SPE, Beijing, China, p. 5. https://doi.org/10.2118/103781ms Kadkhodaie-Ilkhchi, A., Monteiro, S.T., Ramos, F., Hatherly, P., 2010. Rock recognition from MWD Data: A comparative study of boosting, neural networks, and fuzzy logic. IEEE Geosci. Remote Sens. Lett. https://doi.org/10.1109/LGRS.2010.2046312 Kihlman, H., Eriksson, I., Ennis, M., 2010. Robotic Orbital Drilling of Structures for Aerospace Applications, in: SAE Technical Paper Series. https://doi.org/10.4271/2002-01-2636 Kiran, M., Murphy, P., Monga, I., Dugan, J., Baveja, S.S., 2015. Lambda architecture for cost-effective batch and speed big data processing, in: Proceedings - 2015 IEEE International Conference on Big Data, IEEE Big Data 2015. https://doi.org/10.1109/BigData.2015.7364082 Klyuchnikov, N., Zaytsev, A., Gruzdev, A., Ovchinnikov, G., Antipova, K., Ismailova, L., Muravleva, E., Burnaev, E., Semenikhin, A., Cherepanov, A., Koryabkin, V., Simon, I., Tsurgan, A., Krasnov, F., Koroteev, D., 2019. Data-driven model for the identification of the rock type at a drilling bit. J. Pet. Sci. Eng. https://doi.org/10.1016/j.petrol.2019.03.041 Kotsiantis, S.B., Kanellopoulos, D., 2006. Data preprocessing for supervised leaning. Int. J. Comput. Sci. 1, 111–117. https://doi.org/10.1080/02331931003692557 Lee, W.Y., Shih, C.L., Lee, S.T., 2004. Force control and breakthrough detection of a bone-drilling system. IEEE/ASME Trans. Mechatronics. https://doi.org/10.1109/TMECH.2004.823850 Loeken, E.A., Loekkevik, J., 2019. Optimization of an Intelligent Autonomous Drilling Rig: Testing and Implementation of Machine Learning and Control Algorithms for Formation Classification, Downhole Vibrations Management and Directional Drilling. University of Stavanger. Loeken, E.A., Trulsen, A., Holsaeter, A.M., Wiktorski, E., Sui, D., Ewald, R., 2018. Design Principles Behind the Construction of an Autonomous Laboratory-Scale Drilling Rig. IFAC-PapersOnLine. https://doi.org/10.1016/j.ifacol.2018.06.356 Losoya, E.Z., 2016. Real-time Control and Vibrations Analysis of a Completely Automated Miniaturized Rig. Texas A&M University. Losoya, E.Z., Gildin, E., Noynaert, S.F., 2018. Real-time Rate of Penetration Optimization of an Autonomous Lab-Scale Rig using a Scheduled-Gain PID Controller and Mechanical Specific Energy. IFAC-PapersOnLine. https://doi.org/10.1016/j.ifacol.2018.06.355 Luenberger, D.G., 1984. Linear and Nonlinear Programming. Addison-Wesley. Macpherson, J.D., de Wardt, J.P., Florence, F., Chapman, C., Zamora, M., Laing, M., Iversen, F., 2013. Drilling-Systems Automation: Current State, Initiatives, and Potential Impact. SPE Drill. Complet. https://doi.org/10.2118/166263-pa McTiernan, K., 2016. Integration of process optimization and diagnostic functions in a real-time 29

system that supervises the automatic drilling of heterogeneous formation rocks. University of Stavanger, Stavanger. Mostofi, M., Rasouli, V., Mawuli, E., 2011. An estimation of rock strength using a drilling performance model: A case study in blacktip field, Australia. Rock Mech. Rock Eng. https://doi.org/10.1007/s00603-011-0142-9 Moustris, G.P., Hiridis, S.C., Deliparaschos, K.M., Konstantinidis, K.M., 2011. Evolution of autonomous and semi-autonomous robotic surgical systems: A review of the literature. Int. J. Med. Robot. Comput. Assist. Surg. https://doi.org/10.1002/rcs.408 Nadarajah, S., Kotz, S., 2007. Computation of signal-to-noise ratios. Commun. Math. Comput. Chem 57, 105–110. Nilsen, S.J., Hjelm, O.A., 2018. Intelligent Drilling with Sensor Data and Machine-learning. University of Stavanger. Nilsen, U., 2015. Detection of Conglomerate Formations While Drilling. Norwegian University of Science and Technology (NTNU). Norwegian Research Center (NORCE), 2019. An Approach to Autonomous Drilling, Guest Lecture PET575. Parasuraman, R., Sheridan, T.B., Wickens, C.D., 2000. A model for types and levels of human interaction with automation. IEEE Trans. Syst. Man, Cybern. Part ASystems Humans. https://doi.org/10.1109/3468.844354 Pedregosa, F., Gramfort, A., Michel, V., Thirion, B., Grisel, O., Blondel, M., Prettenhofer, P., Dubourg, V., Pedregosa, F., Gramfort, A., Michel, V., Thirion, B., Pedregosa, F., Weiss, R., 2011. Scikitlearn : Machine Learning in Python. J. Mach. Learn. Res. 12, 2825–2830. Ritto, T.G., Soize, C., Sampaio, R., 2010. Robust optimization of the rate of penetration of a drill-string using a stochastic nonlinear dynamical model. Comput. Mech. 45, 415–427. https://doi.org/10.1007/s00466-009-0462-8 Rosenlund, O.H., 2017. Mobile Drilling Robot-A case study of the effects on the construction site. Norwegian University of Science and Technology (NTNU). Russell, S., Norvig, P., 2002. Artificial Intelligence: A Modern Approach (2nd Edition), Prentice Hall. https://doi.org/10.1017/S0269888900007724 Sammut, C., Webb, G.I., Martin, E., Kaski, S., Zheng, F., Zhu, X., Muslea, I., Ting, K.M., Vlachos, M., Miikkulainen, R., Fern, A., Osborne, M., Raedt, L. De, Kersting, Kristian Zeugmann, Thomas Zhang, X., Bain, M., Czumaj, A., Sohler, C., Novak, P.K., 2010. Support Vector Machines, in: Encyclopedia of Machine Learning. Springer US, New York, pp. 941--946. https://doi.org/10.1007/978-0-387-30164-8_804 Schapire, R.E., 2003. The Boosting Approach to Machine Learning: An Overview BT - Nonlinear Estimation and Classification, in: Nonlinear Estimation and Classification. Soendervik, K., 2013. Autonomous Robotic Drilling Systems, in: SPE/IADC Drilling Conference. Society of Petroleum Engineers (SPE), Amsterdam. https://doi.org/10.2118/163466-ms Soroush, H., Rasouli, V., Tokhmechi, B., 2010. A data processing algorithm proposed for identification of breakout zones in tight formations: A case study in Barnett gas shale. J. Pet. Sci. Eng. https://doi.org/10.1016/j.petrol.2010.08.012 Soukup, I.M., Boudreaux, M.S., Ramirez, R., Anderson, J.R., Shim, J., 2017. Improved autodriller 30

performance with direct drillstring Measurements in the drawworks control loop, in: Proceedings - SPE Annual Technical Conference and Exhibition. SPE, San Antonio, Texas, p. 21. https://doi.org/10.2118/187043-MS Sugiura, J., Samuel, R., Oppelt, J., Pastusek, P., Ostermeyer, G.P., John, H., 2015. Drilling Modeling and Simulation: Current State and Future Goals, in: SPE/IADC Drilling Conference and Exhibition. SPE, London, Englend, UK, p. 27. https://doi.org/10.2118/173045-MS Sui, D., Aadnøy, B.S., 2016. Rate of penetration optimization using moving horizon estimation. Model. Identif. Control 37, 149–158. https://doi.org/10.4173/mic.2016.3.1 Teale, R., 1965. The concept of specific energy in rock drilling, in: International Journal of Rock Mechanics and Mining Sciences & Geomechanics Abstracts. pp. 57–73. https://doi.org/10.1016/0148-9062(65)90022-7 Thorogood, J., Aldred, W.D., Florence, F., Iversen, F., 2010. Drilling Automation: Technologies, Terminology, and Parallels With Other Industries. SPE Drill. Complet. https://doi.org/10.2118/119884-pa Wang, P.W., Lin, C.J., 2014. Support vector machines, in: Data Classification: Algorithms and Applications. https://doi.org/10.1201/b17320 Watt, M., Rafati, R., Hamidi, H., 2016. The Application of Robotic Drilling Systems in Extreme Environments. https://doi.org/10.2118/180603-ms Westermann, H., Gorelik, I., Rudat, J., Moritz, C., Neubauer, M., Wallaschek, J., Höhn, O., 2015. A new test rig for experimental studies of drillstring vibrations. SPE Drill. Complet. 30, 119–128. https://doi.org/10.2118/176019-PA Wiktorski, E., Geekiyanage, S.C.H., Løken, E., Sui, D., 2019. Comparative Study of Surface and Downhole Drillstring Vibrations Measurements on a Laboratory-Scale Drilling Rig. https://doi.org/10.2118/195610-ms Wilson, A., 2016a. Drilling-Systems-Automation Roadmap: The Means To Accelerate Adoption. J. Pet. Technol. 67, 137–138. https://doi.org/10.2118/0915-0137-jpt Wilson, A., 2016b. Drilling Modeling and Simulation: Current State and Future Goals. J. Pet. Technol. https://doi.org/10.2118/0915-0140-jpt Wold, S., Esbensen, K., Geladi, P., 1987. Principal component analysis, Chemometrics and intelligent laboratory systems. Elsevier. https://doi.org/10.1016/0169-7439(87)80084-9 Yuan, Y., 2008. Step-sizes for the gradient method. AMS IP Stud. Adv. Math. 42, 785. Zacny, K., Paulsen, G., Szczesiak, M., 2011. Challenges and methods of drilling on the Moon and Mars, in: IEEE Aerospace Conference Proceedings. https://doi.org/10.1109/AERO.2011.5747261

31

APPENDIX

Figure A- 1: An overview of the agent programming methodology according to the data structures used. First, in order to define the rig objects in the digital domain, objects oriented programming style is employed. Then these objects are bought together to perform a predefined set of actions. These actions define the states of the agent; see the example given for defining the drilling state, S3. Finally, all states are linked together into a list to define the agent as an immutable list (i.e., a tuple).

32

Highlights This paper consist of: • • • • •

Proof of concept of a novel autonomous small-scale drilling agent. Agents' innovative software architecture, hardware design and data management. Gradient descent for ROP optimization in WOB-RPM control space for the first time. Abrupt formation changes detection with and without machine learning. Learning outcomes considering industrial drilling automation and digitalization.

Declaration of interests    ☒ The authors declare that they have no known competing financial interests or personal relationships  that could have appeared to influence the work reported in this paper.    ☐The authors declare the following financial interests/personal relationships which may be considered  as potential competing interests: