Monday, January 11, 2010

Eucalyptus Code Reading


Some notes on Eucalyptus source



Architecture

See: http://open.eucalyptus.com/wiki/EucalyptusInstall_v1.6

Clound Controller (CLC)

The most complex component in terms of implementation and nuber of lines of code. Implemented in Java and code is located under directory *clc*. It seems it supports both ant and maven build.

Packages used:
  • mule
Modules:
  • authentication
  • bootstrap
    Built to generate an executable named eucalyptus-cloud which can be used to start up the cloud.
  • cloud
  • cluster-manager
    /edu/ucsb/eucalyptus/cloud/cluster
    Allocator.java
    ClusterAllocator.java
    ClusterEndpoint.java
    ClusterEnvelope.java

    QueuedEvent.java
    QueuedLogEvent.java

    Reservation.java
    Reservations.java
    VmInstance.java
    VmInstances.java

    VmAllocationTransaction.java
    VmTypeAvailability.java
    VmTypeVerify.java
    VmTypes.java

    Callbacks
    UnassignAddressCallback.java
    AssignAddressCallback.java
    ConfigureNetworkCallback.java
    ConsoleOutputCallback.java
    MultiClusterCallback.java
    QueuedEventCallback.java
    RebootCallback.java
    StartNetworkCallback.java
    StopNetworkCallback.java
    TerminateCallback.java
    VmRunCallback.java
    VolumeAttachCallback.java
    VolumeDetachCallback.java

    /edu/ucsb/eucalyptus/cloud/cluster
    AddressManager.java
    CreateVmInstances.java
    StateSnapshot.java
    SystemState.java
    VmAdmissionControl.java
    VmControl.java
    VmMetadata.java
    VmReplyTransform.java


    /com/eucalyptus/cluster
    Cluster.java
    ClusterBootstrapper.java
    ClusterMessageQueue.java
    ClusterNodeState.java
    ClusterState.java
    ClusterThreadGroup.java
    Clusters.java
    Networks.java

    /com/eucalyptus/cluster/handlers
    AbstractClusterMessageDispatcher.java
    AddressStateHandler.java
    ClusterCertificateHandler.java
    LogStateHandler.java
    NetworkStateHandler.java
    ResourceStateHandler.java
    VmStateHandler.java
  • configuration
  • core
    edu/ucsb/eucalyptus/cloud/entities: Includes basic beans which represent various objects in the system (e.g. vmtype, user, bucket, image)
  • distribution
  • dns
  • group-manager
  • image-manager
  • interface
  • key-manager
  • msgs
    com/eucalyptus/bootstrap:
    Depends, Provides, Resource
    Component: each component has a resource provider.
    ResourceProvider
    Bootstrapper
    SystemBootstrapper: includes methods to load and start all bootstrappers contained in jars.
    BootstrapFactory: provides static methods to initialize bootstrappers, configuration resources and resource providers.
    com/eucalyptus/event
    AbstractNamedRegistry.java
    ClockTick.java: clock tick event
    Event.java: event abstraction
    EventListener.java: event listener abstration
    GenericEvent.java: adds message to an event
    ListenerRegistry.java: a event listener registry
    ReentrantListenerRegistry.java
    StateEvent.java: represents an event that has state.
    StatefulNamedRegistry.java
    SystemClock.java: periodically triggers clocktick event.
    com/eucalyptus/util

    edu/ucsb/eucalyptus/msgs/
    Files under this directory defines various message types.
    Configuration.groovy
    DNS.groovy
    EventRecord.java
    Messages.groovy
    StorageController.groovy
    Unimplemented.groovy
    VPN.groovy
    VmAddresses.groovy
    VmBlockDevice.groovy
    VmControl.groovy
    VmImages.groovy
    VmKeys.groovy
    VmLocation.groovy
    VmNetwork.groovy
    VmSecurity.groovy
    Walrus.groovy
  • storage-common
    edu/ucsb/eucalyptus/cloud/ws:
    ChunkedDataFile.java
    CompressedChunkedFile.java

    edu/ucsb/eucalyptus/storage
    StorageManager.java: interface for S3?

    edu/ucsb/eucalyptus/storage/fs:
    FileIO.java
    FileReader.java
    FileSystemStorageManager.java: impl of StorageManager.
    FileWriter.java

  • storage-controller
    com/eucalyptus/bootstrap
    BlockStorageBootstrapper: bootstrappers for block storage.
    edu/ucsb/eucalyptus/cloud/ws
    BlockStorage: EBS (to read). It uses StorageManager from storage-common as underlying impl.
    StorageEventListener
    /edu/ucsb/eucalyptus/storage/
    AOEManager.java: implements StorageExportManager
    BlockStorageChecker.java
    BlockStorageManagerFactory.java: factory for LVM2Manager.
    LVM2Manager.java: impls LogicalStorageManager
    LogicalStorageManager.java: interface for logical volume management
    StorageExportManager.java: interface for export of volume
    /edu/ucsb/eucalyptus/ic
    StorageController.java: handles messages.

  • walrus (S3 impl)
    edu/ucsb/eucalyptus/cloud/ws:
    ObjectReader.java
    TorrentClient.java
    TorrentCreator.java
    Torrents.java
    Tracker.java
    WalrusControl.java
    WalrusEventListener.java: event listener. Currently, listens to StopComponentEvent.
    WalrusImageManager.java
    WalrusManager.java
    WalrusBlockStorageManager.java
    WalrusStatistics.java
    edu/ucsb/eucalyptus/ic:
    Walrus.java: handles messages.

  • wsstack
  • www

Cluster Controller (CC)

It interacts with two parts: cloud controller and node controllers. Its interface is described in file wsdl/eucalyptus_cc.wsdl. Axis2/C is used to generate C stub functions from wsdl.
It has two parts in the code: client and server. It provides services using SOAP-based web service.

client

It relies on
  • generated CC client stub functions from *wsdl/eucalyptus_cc.wsdl*
Code:
cluster/CCclient.c
cluster/cc-client-marshal-adb.c (main file)
cluster/cc-client-marshal.h
Client functions make use of stub functions generated from wsdl using axis2/C to construct SOAP request messages. The generated tool (by make) can be used to interact with cluster controller services.

server

When you build this part, a shared object is generated and deployed to Axis2/C environment. The axis2/C service name is "EucalyptusCC".
To fulfill the requests, sometimes it needs to interact with node controller (e.g. get information of the instances running on a specific node). So this part relies on generated stubs from both wsdl/eucalyptus_cc.wsdl and wsdl/eucalyptus_nc.wsdl. It acts as a server to provide services for external parties while it also acts as a service client to interact with other services.
So, it relies on
  • generated NC client stub functions *wsdl/eucalyptus_nc.wsdl*
  • generated CC server stub functions *wsdl/eucalyptus_cc.wsdl*
Code:
cluster/handlers.c
cluster/handlers.h
cluster/server-marshal.c
cluster/server-marshal.h
Functions in *handlers.c* do the real work when a service request is received. If necessary, it sends service requests to node controller.
File *server-marshal.c* extracts the request parameters using generated stub functions and invokes corresponding functions in file *handlers.c*. So it acts like a mediator which converts web service request to function name and parameters.

Storage Controller

Provides storage services for other parts. However, the form is not web service, just functions. Other parts directly call storage controller functions.
Code
  • storage/walrus.c, storage/walrus.h
    Provides functions:
    int walrus_object_by_url (const char * url, const char * outfile, const int do_compress);
    int walrus_object_by_path (const char * path, const char * outfile, const int do_compress);
    int walrus_image_by_manifest_url (const char * url, const char * outfile, const int do_compress);
    int walrus_image_by_manifest_path (const char * manifest_path, const char * outfile, const int do_compress);
    int walrus_verify_digest (const char * url, const char * digest_path);
    These functions are used to interact with walrus services (using REST interfaces instead of SOAP-based interfaces).
  • storage/storage.c, storage/storage.h
    provides higher-level abstraction which eases interaction of application with storage system.
    It is used by Node Controller and Cluster controller(? it is included as header file, but seems none of its functions are used).

A executable named Wclient is generated when you "make all".

Node Controller (NC)

Node controller manages the instances running on a physical machine. It also provides services in the form of web services.
Its interface is described in file wsdl/eucalyptus_nc.wsdl

Server

Provides various services to other parts.
Code:
node/handlers.c
node/handlers_default.c
node/handlers.h
node/handlers_kvm.c
node/handlers_xen.c
node/server-marshal.c
node/server-marshal.h
Similar to implementation of cluster controller, files *handlers.c* does the real work to interact with virtual machine monitoring. Currently, both kvm and xen are supported. The concrete work involved to interact with a specific type of virtualization implementation is delegated to functions in file handlers_kvm.c, handlers_xen.c or handlers_default.c based on the user configuration. Under the hood, libvirt is used to interact with virtualization implementation.
server-marshal.c just extracts information from the incoming service requests and invokes corresponding functions in file *handlers.c*.
When a service needs to interact with storage system (e.g. download an image, create an image), it makes use of Storage Controller.

Client

Access services provided by NC server.
Code:
node/client-marshal-adb.c
node/client-marshal.h
node/client-marshal-local.c
node/NCclient.c
NCclient.c contains the main function. It parses the command and dispatch the requests to corresponding functions in file *client-marshal-adb.c* or *client-marshal-local.c*.
If the service is not local, client-marshal-adb.c is used. Basically the request is transformed to SOAP message and sent to remote NC service.
If the service is local, client-marshal-local.c is used. The corresponding implementation functions are directly invoked without hassle of web service stuff.

Misc.

network

See http://open.eucalyptus.com/wiki/EucalyptusNetworking_v1.6 for networking configuration.
In managed mode,
Manipulates the dhcpd configuration file and use "brctl" command from package *bridge-utils*. Also it supports VLAN.

gatherlog

Another sets of services described in file wsdl/eucalyptus_gl.wsdl. Code is located under directory *gatherlog*.

util

tools



2 comments:

Long Thai said...

Hi there.
I'm trying to read Eucalyptus source code for my graduate thesis but it's really confusing. Do you have time to explain more about them? Or, can you give me some suggestion about reading them? Thank you very much.
Regards.
Long

Unknown said...

hello
i need source code detailed information too
thank you if help me
omid aslambakhsh