Eucalyptus Code Reading

Some notes on Eucalyptus source

1. Architecture
2. Clound Controller (CLC)
3. Cluster Controller (CC)
   1. client
   2. server
4. Storage Controller
5. Node Controller (NC)
   1. Server
   2. Client
6. Misc.
   1. network
   2. gatherlog
   3. util
   4. tools

Architecture

See: http://open.eucalyptus.com/wiki/EucalyptusInstall_v1.6

Clound Controller (CLC)

The most complex component in terms of implementation and nuber of lines of code. Implemented in Java and code is located under directory *clc*. It seems it supports both ant and maven build.

Packages used:

• mule
  

Modules:

• authentication
• bootstrap
  Built to generate an executable named eucalyptus-cloud which can be used to start up the cloud.
  
• cloud
• cluster-manager
  /edu/ucsb/eucalyptus/cloud/cluster
  
  Allocator.java
  ClusterAllocator.java
  ClusterEndpoint.java
  ClusterEnvelope.java
  
  QueuedEvent.java
  QueuedLogEvent.java
  
  Reservation.java
  Reservations.java
  VmInstance.java
  VmInstances.java
  
  VmAllocationTransaction.java
  VmTypeAvailability.java
  VmTypeVerify.java
  VmTypes.java
  
  Callbacks
  UnassignAddressCallback.java
  AssignAddressCallback.java
  ConfigureNetworkCallback.java
  ConsoleOutputCallback.java
  MultiClusterCallback.java
  QueuedEventCallback.java
  RebootCallback.java
  StartNetworkCallback.java
  StopNetworkCallback.java
  TerminateCallback.java
  VmRunCallback.java
  VolumeAttachCallback.java
  VolumeDetachCallback.java
  
  
  /edu/ucsb/eucalyptus/cloud/cluster
  
  AddressManager.java
  CreateVmInstances.java
  StateSnapshot.java
  SystemState.java
  VmAdmissionControl.java
  VmControl.java
  VmMetadata.java
  VmReplyTransform.java
  
  
  
  /com/eucalyptus/cluster
  
  Cluster.java
  ClusterBootstrapper.java
  ClusterMessageQueue.java
  ClusterNodeState.java
  ClusterState.java
  ClusterThreadGroup.java
  Clusters.java
  Networks.java
  
  
  /com/eucalyptus/cluster/handlers
  
  AbstractClusterMessageDispatcher.java
  AddressStateHandler.java
  ClusterCertificateHandler.java
  LogStateHandler.java
  NetworkStateHandler.java
  ResourceStateHandler.java
  VmStateHandler.java
• configuration
• core
  edu/ucsb/eucalyptus/cloud/entities: Includes basic beans which represent various objects in the system (e.g. vmtype, user, bucket, image)
  
• distribution
• dns
• group-manager
• image-manager
• interface
• key-manager
• msgs
  com/eucalyptus/bootstrap:
  
  Depends, Provides, Resource
  Component: each component has a resource provider.
  ResourceProvider
  Bootstrapper
  SystemBootstrapper: includes methods to load and start all bootstrappers contained in jars.
  BootstrapFactory: provides static methods to initialize bootstrappers, configuration resources and resource providers.
  
  com/eucalyptus/event
  
  AbstractNamedRegistry.java
  ClockTick.java: clock tick event
  Event.java: event abstraction
  EventListener.java: event listener abstration
  GenericEvent.java: adds message to an event
  ListenerRegistry.java: a event listener registry
  ReentrantListenerRegistry.java
  StateEvent.java: represents an event that has state.
  StatefulNamedRegistry.java
  SystemClock.java: periodically triggers clocktick event.
  
  com/eucalyptus/util
  
  edu/ucsb/eucalyptus/msgs/
  Files under this directory defines various message types.
  
  Configuration.groovy
  DNS.groovy
  EventRecord.java
  Messages.groovy
  StorageController.groovy
  Unimplemented.groovy
  VPN.groovy
  VmAddresses.groovy
  VmBlockDevice.groovy
  VmControl.groovy
  VmImages.groovy
  VmKeys.groovy
  VmLocation.groovy
  VmNetwork.groovy
  VmSecurity.groovy
  Walrus.groovy
  
• storage-common
  edu/ucsb/eucalyptus/cloud/ws:
  
  ChunkedDataFile.java
  CompressedChunkedFile.java
  
  
  edu/ucsb/eucalyptus/storage
  
  StorageManager.java: interface for S3?
  
  
  edu/ucsb/eucalyptus/storage/fs:
  
  FileIO.java
  FileReader.java
  FileSystemStorageManager.java: impl of StorageManager.
  FileWriter.java
  
  
  
• storage-controller
  com/eucalyptus/bootstrap
  
  BlockStorageBootstrapper: bootstrappers for block storage.
  
  edu/ucsb/eucalyptus/cloud/ws
  
  BlockStorage: EBS (to read). It uses StorageManager from storage-common as underlying impl.
  StorageEventListener
  
  /edu/ucsb/eucalyptus/storage/
  
  AOEManager.java: implements StorageExportManager
  BlockStorageChecker.java
  BlockStorageManagerFactory.java: factory for LVM2Manager.
  LVM2Manager.java: impls LogicalStorageManager
  LogicalStorageManager.java: interface for logical volume management
  StorageExportManager.java: interface for export of volume
  
  /edu/ucsb/eucalyptus/ic
  
  StorageController.java: handles messages.
  
  
  
• walrus (S3 impl)
  edu/ucsb/eucalyptus/cloud/ws:
  
  ObjectReader.java
  TorrentClient.java
  TorrentCreator.java
  Torrents.java
  Tracker.java
  WalrusControl.java
  WalrusEventListener.java: event listener. Currently, listens to StopComponentEvent.
  WalrusImageManager.java
  WalrusManager.java
  WalrusBlockStorageManager.java
  WalrusStatistics.java
  
  edu/ucsb/eucalyptus/ic:
  
  Walrus.java: handles messages.
  
  
  
• wsstack
• www
  
  

Cluster Controller (CC)

It interacts with two parts: cloud controller and node controllers. Its interface is described in file wsdl/eucalyptus_cc.wsdl. Axis2/C is used to generate C stub functions from wsdl.
It has two parts in the code: client and server. It provides services using SOAP-based web service.

client

It relies on

• generated CC client stub functions from *wsdl/eucalyptus_cc.wsdl*

Code:

cluster/CCclient.c
cluster/cc-client-marshal-adb.c (main file)
cluster/cc-client-marshal.h

Client functions make use of stub functions generated from wsdl using axis2/C to construct SOAP request messages. The generated tool (by make) can be used to interact with cluster controller services.

server

When you build this part, a shared object is generated and deployed to Axis2/C environment. The axis2/C service name is "EucalyptusCC".
To fulfill the requests, sometimes it needs to interact with node controller (e.g. get information of the instances running on a specific node). So this part relies on generated stubs from both wsdl/eucalyptus_cc.wsdl and wsdl/eucalyptus_nc.wsdl. It acts as a server to provide services for external parties while it also acts as a service client to interact with other services.
So, it relies on

• generated NC client stub functions *wsdl/eucalyptus_nc.wsdl*
• generated CC server stub functions *wsdl/eucalyptus_cc.wsdl*

Code:

cluster/handlers.c
cluster/handlers.h
cluster/server-marshal.c
cluster/server-marshal.h

Functions in *handlers.c* do the real work when a service request is received. If necessary, it sends service requests to node controller.
File *server-marshal.c* extracts the request parameters using generated stub functions and invokes corresponding functions in file *handlers.c*. So it acts like a mediator which converts web service request to function name and parameters.

Storage Controller

Provides storage services for other parts. However, the form is not web service, just functions. Other parts directly call storage controller functions.
Code

• storage/walrus.c, storage/walrus.h
  Provides functions:
  
  int walrus_object_by_url (const char * url, const char * outfile, const int do_compress);
  int walrus_object_by_path (const char * path, const char * outfile, const int do_compress);
  int walrus_image_by_manifest_url (const char * url, const char * outfile, const int do_compress);
  int walrus_image_by_manifest_path (const char * manifest_path, const char * outfile, const int do_compress);
  int walrus_verify_digest (const char * url, const char * digest_path);
  
  These functions are used to interact with walrus services (using REST interfaces instead of SOAP-based interfaces).
  
• storage/storage.c, storage/storage.h
  provides higher-level abstraction which eases interaction of application with storage system.
  It is used by Node Controller and Cluster controller(? it is included as header file, but seems none of its functions are used).
  

A executable named Wclient is generated when you "make all".

Node Controller (NC)

Node controller manages the instances running on a physical machine. It also provides services in the form of web services.
Its interface is described in file wsdl/eucalyptus_nc.wsdl

Server

Provides various services to other parts.
Code:

node/handlers.c
node/handlers_default.c
node/handlers.h
node/handlers_kvm.c
node/handlers_xen.c
node/server-marshal.c
node/server-marshal.h

Similar to implementation of cluster controller, files *handlers.c* does the real work to interact with virtual machine monitoring. Currently, both kvm and xen are supported. The concrete work involved to interact with a specific type of virtualization implementation is delegated to functions in file handlers_kvm.c, handlers_xen.c or handlers_default.c based on the user configuration. Under the hood, libvirt is used to interact with virtualization implementation.
server-marshal.c just extracts information from the incoming service requests and invokes corresponding functions in file *handlers.c*.
When a service needs to interact with storage system (e.g. download an image, create an image), it makes use of Storage Controller.

Client

Access services provided by NC server.
Code:

node/client-marshal-adb.c
node/client-marshal.h
node/client-marshal-local.c
node/NCclient.c

NCclient.c contains the main function. It parses the command and dispatch the requests to corresponding functions in file *client-marshal-adb.c* or *client-marshal-local.c*.
If the service is not local, client-marshal-adb.c is used. Basically the request is transformed to SOAP message and sent to remote NC service.
If the service is local, client-marshal-local.c is used. The corresponding implementation functions are directly invoked without hassle of web service stuff.

Misc.

network

See http://open.eucalyptus.com/wiki/EucalyptusNetworking_v1.6 for networking configuration.
In managed mode,
Manipulates the dhcpd configuration file and use "brctl" command from package *bridge-utils*. Also it supports VLAN.

gatherlog

Another sets of services described in file wsdl/eucalyptus_gl.wsdl. Code is located under directory *gatherlog*.

util

tools