Abstract
Linux containers is a very important and useful system because of the need for few resources consumption, fast startup times, and high I/O performance especially when it’s compared to virtual machines (VMs) on hypervisors, in multi-tenant environments. This paper is focused on how Linux containers can be secured from attackers who enforced through software kernel mechanisms, by using Intel SGX technology from outside attacks on Docker. This mechanism is called SCONE. SCONE is useful because Linux containers (which are managed by Docker or Kubernetes) do not provide guarantees concerning the security of application data within containers.
SCONE offers a secure C standard library interface that encrypts/decrypts I/O data in order to diminish the execution repercussion of thread synchronization and system calls within SGX enclaves and a minimized trusted computing base (TCB). SCONE also supports user-level threading and asynchronous system calls. This paper through the analysis and studying shows an evaluating that it protects unmodified applications with SGX. Many multi-tenant environments use LinuX Containers (LXC) in order to confer performance seclusion of applications. The Docker Swarm is used for development (or Kubernetes) and Docker for packaging of containers. Currently, virtualization container-based is widely used and has become trend.
However, containers seem to have better performance compared to improved hardware virtualization (VM) in hypervisor. Containers provide faster start up time, better I/O throughput and better delay. They undoubtedly provide uncertain security benefits than virtual machines (VMs). This is due because of the fact that there is the need for the host operating system (OS) kernel to protect a larger interface and usually uses only software mechanisms for isolation. Mainly, container seclusion mechanism what they actually do, is to protect the environment from unreliable containers. Tenants from the other side, are protecting not only unreliable mechanisms but also are providing covertness and probity for their application data from unauthorized access from higher-privileged system software as well as the core of the operating system (OS) and the supervisor.
Malicious users are primarily looking for unprotected spots in virtualized system software for attacking or maybe they are staking at the authorized administration accreditations. Lately, was achieved the hardware mechanism to protect in user-level software from the widely preferential system software. Specifically, in 2015 Intel promoted to the market the Software Guard eXtensions (SGX) for their own CPUs. The Software Guard eXtensions (SGX) provide support for secure enclaves.
The function of an enclave is to protect application codes and data from entry by other software and to provide higher-privileged software. Into the enclave page cache memory (EPC), there are memory pages which correspond to an enclave, in which there is no possibility for access using code outside of the enclave. So, SGX is recommended as an ideal choice for the security of containers, covertness and probity of the data are assured because the application process of a container can be executed inside an enclave. The SGX mechanism which is providing security in containers is required to cope up two basic problems. The first problem which this mechanism is facing up is the improving the size of the trusted computing base (TCB) inside an enclave and the protection of existing applications into secure containers. The second problem that might need to face up is the challenge of conservation of low burden overhead in for security of containers by exploiting the possibilities of SGX mechanism.
The trusted computing base (TCB) is a factor which employ in general because of it size. According to other studies the TCB occupies huge size in Windows applications where is executed into enclaves, because of the libraries and the operating system (OS). An appropriate protection for TCB is necessary because otherwise a malicious user may entry into the application data or jeopardize the confidentiality, for that reason is rendered necessary to keep a minimized size of a container’s TBC into an enclave. A problem that arises from the use of enclaves is its general attribution which resulting by the OS kernel which is not trustworthy. This because of enclave code which cannot implement system calls. The function of an enclave thread is to copy arguments from memory and abandon the enclave before a system call.
The transfer of those threads becomes very costly. The reason for this is that these threads perform saving and restoring the enclave’s carry out situation. One more reason which is responsible for the delay except from cache misses, is cache lines which is necessary to be decrypted when transferred from memory. In addition, out of the enclave page cache (EPC) provoke costly page errors. In order to keeping up a minimized TCB size for secure containers, firstly is notable that containers typically perform network duty (e.g. NGINX, Redis, Memcached and Apache) therefore, is considered essential small interface for backing of the system.
The communication is achieved by the outside through network reception or stdin/ stdout streams, secluded or evanescent file systems are used, also do not enter other I/O devices directly. For lightening of secure containers, the enclave code can enter into the memory outside of the enclave without the burden of the performance. Nevertheless, remains costly the overhead of access (whether for leaving or entering) the enclave high system calls frequency in applications. In this paper SCONE is mostly presented.
The Secure CONtainer Environment (SCONE) which is for Docker uses the Software Guard eXtensions SGX mechanism in order to run Linux applications in secure containers. SCONE mechanism contains various benefits that are necessary. The first benefit of the SCONE is the minimized size of the TCB. SCONE is providing a C standard library interface for container’s application. The C standard library is programmed with static linking way (with the use of libC library) inside the enclave.
In addition, System calls are safe despite they are carry out out of the enclaves because of the programmed function of encryption -decryption. This function is done based on the description of every file. In more details for that benefit is that except from the encryption of the files that are being out of the enclaves, the network transportation is also secured by transport layer security (TLS). Evanescent semantic file system is supplied by SCONE. The second benefit is related with the frequent movement of enclave threads and the resulting outcome is that it causes a long delay which is not desirable.
Here, it comes the solution by SCONE mechanism again. SCONE give the ability to increase the time while threads are into the enclave by a threading implementation which is also provided. SCONE provide a mode of operation so as there is no problem with the thread synchronization deviation. In order to do this, SCONE corresponds the application threads with operation system (OS) threads into an enclave modifying OS threads and application threads. By using of SCONE there is no longer the need for enclave threads to be on the outside part of the enclave. To make this happen carries out asynchronous system calls where OS threads out part of the enclave perform system calls.
SCONE mechanism also is minimizing the cost of frequent memory access from the enclaves. The cause is the encryption of the application data (e.g. network buffers, cached files). The third benefit of SCONE mechanism is based on Docker and secure containers. Into the Docker engine the secure containers looks like common containers.
Container images are reliable because are created from specialists, so non consultant users can easily can advantage of SCONE because it provide security in container image. For secure containers SCONE necessity need the Intel CPUs with SGX-enabled. The SGX provide a driver and an optional kernel in order to do asynchronous system calls. Subsequently, an experimental evaluation of SCONE mechanism on SGX will carried out in order to find out performance of well-known software (e.g. Memcached, Redis, NGINX) which is 0,6-1,2 optimization and 0,6–2 code size, although there are execution limitations in implementations.
In general, SCONE mechanism produces those benefits where were mentioned above more specifically asynchronous system calls, security in containers and encrypted TLC user connections. This paper is structured as follows. In Section II is mostly the theoretical background for secure Linux containers, SGX mechanism and TCB that are used in SCONE mechanism. Section III deals with SCONE mechanism and how it works. In Section IV focuses at the Evaluation of the SCONE mechanism and the way of it approach.
Section V details at the Related Works. Final Section VI concludes this survey emphasizing on highlights of this paper. II. SECURITY IN LINUX CONTAINERS The target of this paper is to study in depth the way that security in container may supervene and to confirm probity and trustiness of the system of the memory, code, implementation out of the enclaves and network I/O. This is very important because there is the need for protection of the malicious users. A. LinuX Containers (LXC) LinuX Containers (LXC) is an operation system (OS) virtualization method. In this method multiple isolated containers (Linux systems) are executed in a central computer with a Linux kernel.
This method is well-known for packaging, development and domineer different services ( e.g. web servers). In Virtual Machines (VMs) is known that they require hypervisor or some special operating system kernel, this does not happen in Linux containers. Actually, Linux containers are based on some properties of Linux kernel in order to insulate some processes. In this way is achieved no worries for system calls or need for hardware devices.
What comes out is that now processes into the container can perform like normal processes, despite some characteristics of the process (e.g. file systems) may cause delay. Linux containers in general offer a functional environment. They use a central operating system for I/O services and for the resources. LinuX Containers (LXC) and Docker by using some special characteristics of Linux kernel, C interfaces and namespaces, produce the containers.
The namespace characteristic of Linux kernel is responsible for the right resource management. Actually, a parent produce a child which has a specific number of resource and a root file system virtual network devices. Insulation execution into the containers offers the C interface characteristic, by using a scheduler which there is into the Linux kernel. Kubernetes and Docker Swarm frameworks need for composition and implementation of containers, in order to produce and regulate the association between the containers in a cluster. Architectures of a micro-service are as follows.
There are containers which are occupy a small number of resources and a well organized network interfaces. B. Intel Software Guard eXtensions (SGX) The Intel CPU’s Software Guard eXtensions (SGX) donate the ability to the application to confirm the privacy and the probity from the risk of operating system, hypervisor to be stake. This mechanism also provides security from malicious users with physical access, under the condition that the CPU is not ruptured. SGX mechanism produce enclaves which are secure working out environments.
Data and code of enclaves are in physical memory in order to be protected which is named Enclave Page Cache (EPC). Data and code of cache are well protected because of the CPU access controls. Data are transferred as it was mentioned before in EPC pages with the security of cache lines in order to move to DRAM. For the guarantee of the transport of the data there is an on chip Memory Encryption Engine (MEE) which encrypts -decrypts cache lines into the enclave page cache in DRAM. In addition, the term probity means the protection of reset and alternation of the memory thanks to the ability of the detection of this mechanism.
The code out of the enclaves has not the ability of access in enclave page cache (EPC), in contrast with the code in enclaves which can enter in DRAM from enclave memory, to promote function call parameters and results. Besides it is duty of enclave code to check the probity of all data. • Enclaves are insulted memory regions of code and data that are highly protected. The code of enclave use ECREATE instruction which define an SGX enclave control structure (SECS) in the Enclave Page Cache (EPC).
With the EADD instruction adds secured pages to the enclave. The SGX mechanism register this addition the new enclave, its permissions and virtual address. After this produce protection constraints (e.g. produce the securing of the enclave corresponds the page to virtual address in which can enter). After page loading the EINIT instruction is carries out encryption which used from distance for confirmation.
The SGX mechanism also provide a paging mechanism to switch pages in enclave page cache and in DRAM in order to keep the enclave applications which require more memory. More specifically this system provides some special instructions to force hardware to copy a page in some secure buffer with ability of encryption out of EPC in DRAM. In case of reusing the EPC memory, need to flush TLB entries by constraining the hardware to use this protocol. • Threading is supported by SGX. SGX mechanism maintain multi-threaded implementation into the enclaves.
Every enclave of a thread execution is in a 4 KB thread control structure (TCS). For all these some instructions are necessary. The first instruction is EENTER where allow execution of the enclave code which is swapping the CPU to enclave mode also go over to a definition of an enclave offset. In contrast to the other instruction EEXIT which forces a thread to abandon an enclave. • Performance Overhead is provided by SGX when executing enclave code. The factors that cause this performance overhead are three.
The first factor is that specific instructions cannot be executed into the enclave. Those thread should execute out of the enclave before the system calls. These transports of threads are in impact to the general performance for security reasons. A number of checks and updates need to execute and a TLB flush too. Is also necessary to copy the memory based enclave arguments between DRAM and EPC.
The second factor for the performance overhead is recordings in memory and cache misses of enclave’s code thanks to the MME which performs encryption tasks cache in lines. The third and last factor for this overhead is when applications need more memory than this which is available in EPC. To solve this problem is necessary to do switching in pages between trusted and untrusted memory. By doing this incur is caused (swapping pages in memories) because earlier need to be encryption.
For the protection virtual address from attackers there is a eviction protocol which stop all enclave threads and flushes the TLB. C. Design trade-offs In order to have security in Linux Containers with by using SGX mechanism is necessary to form system support which is be placed into the enclaves to allow the security in Linux processes in a container. For this system is necessary to take into consideration two affects. The first affect is related to security features of containers, to minimize the TBC size and unprotected interface. The second affect is related to the delay of the performance because of containments of SGX.
For the solution of those affect that were mentioned above and cope to SCONE mechanism need to look into similar mechanisms. • External container interface need to use C standard library (libc) in order to perform processes into the secure containers. C library need to execute in system calls. System calls cannot execute into the enclaves.
An interface out of the enclaves, is necessary a secure container to exhibit, to the central operating system. The central operating system is not secure region for that reason external interface is becoming a target from malicious users. For this reason, this design got some security problems. For example, if a malicious user violates the central operating system can have access to the interface and processes into the enclaves might be in danger. Key role in the design the TCB which is basic for execution the interface in the enclave and external interface. • System call overhead is very common in all secure Linux containers designs.
Systems calls cause delay because they are executed out of the enclaves. Services that use containers with a high system call frequency bear as a result reverberation in performance (e.g. network services). In order to check the quantity of this overhead of frequent system call, study is conducted on an Intel Xeon CPU at 3.4 GHz which pwrite system calls were executed with enclave and with no enclave.
The implementation is conducted in using the Intel SGX SDK for Linux environment where execute system calls by using threads abandonment and entering again into the enclave. Threads are differentiated and the pwrite buffer size. • Memory access overhead is an impact from the task of secure container’s frequent access in enclave pages. A study is conducted comparing this overhead with the Linux SGX SDK at the same hardware.
The measurement of the study shows the time sequential and random read/write operations normalized contrary an implementation with no enclave. In general, all operations are working out up to 256 MB under the condition of access differently-sized memory regions. In conclusion, from this study the result is that there is the need for a secure container design which will minimize access to enclave page cache memory. If there wasn’t any risk a good choice would be to choose the DRAM (untrusted memory) much more but by providing the integrity of data.
III. SCONE MECHANISM The main goal of this study is to provide security in containers on the top of an untrusted operating system. Actually, services which are containerized need to protect by attackers from secure containers. Also, secure containers need to suit in Docker’s container environments. In order to do so, it is necessary to secure container images by system administrators using Docker in a trusted environment (e.g.
EPC memory, operating system) and execute secure containers in an untrusted environment (e.g. DRAM memory). A. Architecture The Secure CONtainer Environment (SCONE) design offers a an interface which is rely on system calls to the central operation system (OS), which is proteced from malicious users. SCONE mechanism also execute logic checks and copies all based on memory returning values into the enclave while initially arguments have go through the application. This is similar of what was mentioned above that is operating system’s kernel which has the feature of protection of malicious users.
SCONE also offers encryption and authentication of data in order to protect probity and trustness of data that were processed through files descriptors. In the figure 1. below there is an overview of the SCONE architecture. In order to avoid the enclave’s transitions (where cause performance overhead) that are not necessary SCONE mechanism is proving threading implementation. Enclave bound application threads are multiplexed across operating system’s (OS) threads. SCONE mechanism is configured is configured in such a way it can manage systems calls of the systems.
Specifically, when a system call is incuredd by an application thread, SCONE is checking if some other application thread that it is available and can execute until the result of the system call is available. As it was mentioned before SCONE mechanism can handle system calls. This feature is called asynchronous system calling. So SCONE produce container processes with asynchronous system call interface to the central operating system.
SCONE’s implementation use a shared memory for copying system call argument and returning the results back and to signal the implementation of the system call. Into the SCONE kernel module system calls are implemented by individual threads running. Therefore, threads that are existing into the enclaves there is no need to be out of the enclave when system calls executed. SCONE mechanism also, is related with existing Docker container environments, and guarantee that secure containers are suitable for the Linux containers standard.
The cantra l operating system need to use a Linux SGX driver in order to increase the performance, a SCONE kernel module. Is remarkable that, SCONE mechanism does not need any functionality from the Intel Linux SDK except from the Linux SGX driver.
