DEEP-2 Rosetta

The DEEP - HybridDataCloud project is pleased to announce the general availability of its second public software release, codenamed Rosetta

Highlights

Key technical highlights:

  • Alien4Cloud-DEEP plugin * Support for the OpenStack normative TOSCA types, for the IndigoDC TOSCA custom types, for DEEP-OC templates * Outputs retrieval and display from the Indigo Orchestrator * IAM authentication support

  • DEEP-Hybrid-DataCloud Cloud Information Provider * Support for multitenancy in CMDBv1, for new Mesos providers, for fetching GPU information from OpenStack and Mesos

  • CloudProviderRanker

    • new release featuring a complete code re-engineered using Spring Boot framework and modified ranking algorithm
  • DEEPaaS API

    • New V2 version of the API, major change that is not backward compatible, supporting async requests, following

    Swagger UI 3.X Specifications, with new options allowing to specify the number of workers that will be used for model operations

  • Infrastructure Manager (IM)

    • new version with multiple bug fixes and enhancements, improving deployment of hybrid clusters.
  • INDIGO Virtual Router Installer

    • new verion with many bug fixes and improvements, supporting OpenStack
  • Monitoring Framework

    • first release through the DEEP-HDC project, providing improved versions of Openstack and Mesos probes and new QCG probe
  • PaaS Orchestrator

    • new version that add support for compute GPU, flavors and for vRouter deployment
  • PaaS Orchestrator Dashboard

    • first stable release with INDIGO-IAM authentication, allowing easier management of infrastructures deloyments
  • Spring Social OIDC

    • first release in DEEP-HDC allowing for the integration of the orchestrator functionality into Spring

    projects such as Alien4Cloud through Spring Social

  • TOSCA types and templates

    • new types for QCG jobs and DEEPaaS, support for Marathon secrets and hybrid deployments
  • udocker

    • many new features, improvements and bug fixes, like the improved support of singularity,

    added support for sha512 hashes, safer parsing of config files and removal of directories, improved installation

Release Notes

The DEEP-2/Rosetta release consists in 10 Products and a number of technical guides:

  • 22 packages and tarballs, for CentOS 7, Ubuntu 16.04, Ubuntu 18.04
  • 6 Docker containers

You can find in the later sections the full list of products, with detailed release notes and instructions for their installation/configuration.

Installation Notes

All DEEP - HybridDataCloud products are distributed from standard Operating Systems (OS) repositories and DockerHub registry of the indigodatacloud and deephdcgo organization.

The packages repositories have the following structure:

All packages in production and preview repositories are signed with the INDIGO - DataCloud gpg key. The public key can be downloaded from here, and the fingerprint from here.

Please import the key BEFORE starting!

  • for CentOS7 save the key under /etc/pki/rpm-gpg/
# rpm --import https://repo.indigo-datacloud.eu/repository/RPM-GPG-KEY-indigodc
  • for Ubuntu:
# wget -q -O - https://repo.indigo-datacloud.eu/repository/RPM-GPG-KEY-indigodc | sudo apt-key add -

Configuring the use of DEEP-HDC repositories

YUM & APT configuration files are available at:

or use the deephdc-release package to install DEEP-HDC repositories:

  • CentOS7:
# wget https://repo.indigo-datacloud.eu/repository/deep-hdc/production/2/centos7/x86_64/base/deephdc-release-2.0.0-1.el7.noarch.rpm
# yum localinstall -y deephdc-release-2.0.0-1.el7.noarch.rpm
  • Ubuntu 16.04:
# wget https://repo.indigo-datacloud.eu/repository/deep-hdc/2/ubuntu/dists/xenial/main/binary-amd64/deephdc-release_2.0.0-1_amd64.deb
# dpkg -i deephdc-release_2.0.0-1_amd64.deb
  • Ubuntu 18.04:
# wget https://repo.indigo-datacloud.eu/repository/deep-hdc/2/ubuntu/dists/bionic/main/binary-amd64/deephdc-release_2.0.0-1_amd64.deb
# dpkg -i deephdc-release_2.0.0-1_amd64.deb

These packages will install required dependencies, the INDIGO - DataCloud public key and ensures the precedence of DEEP-HybridDataCloud repositories over EPEL and Ubuntu.

It is strongly recommended the use of the lastest version of the deephdc-release packages containing the public key and the YUM and APT repository files.

Enable the DEEP-HDC Containers repositories

On the DockerHub Registry, DEEP - HybridDataCloud uses the INDIGO - DataCloud and DEEP-HDC Organizations:

Containers present in those repositories and released in DEEP-1 major release are tagged with “DEEP-2” tag and signed, leveraging the Docker’s trust features so that users can pull trusted images.

Currently, content trust is disabled by default. You must enable it by setting the DOCKER_CONTENT_TRUST environment variable, like below:

export DOCKER_CONTENT_TRUST=1

For more details regarding the “Content Trust in Docker” please read Docker’s Documentation

Content trust is associated with the TAG portion of an image. For DEEP-2 (Rosetta) release the signed tag is DEEP-2. See example bellow if you want to ensure the correct use of DEEP-HDC images:

$ export DOCKER_CONTENT_TRUST=1

$ docker pull indigodatacloud/orchestrator:2.2.0-FINAL
No trust data for 2.2.0-FINAL

$ docker pull indigodatacloud/orchestrator:DEEP-2
Pull (1 of 1): indigodatacloud/orchestrator:DEEP-2@sha256:441c8b037684422ccdf2fdec322c9c09904ed3ce74d9fcc7d2862a9f53ad36be
sha256:441c8b037684422ccdf2fdec322c9c09904ed3ce74d9fcc7d2862a9f53ad36be: Pulling from indigodatacloud/orchestrator
93857f76ae30: Pull complete
[...]
e8c92b40b492: Pull complete
Digest: sha256:441c8b037684422ccdf2fdec322c9c09904ed3ce74d9fcc7d2862a9f53ad36be
Status: Downloaded newer image for indigodatacloud/orchestrator@sha256:441c8b037684422ccdf2fdec322c9c09904ed3ce74d9fcc7d2862a9f53ad36be
Tagging indigodatacloud/orchestrator@sha256:441c8b037684422ccdf2fdec322c9c09904ed3ce74d9fcc7d2862a9f53ad36be as indigodatacloud/orchestrator:indigo_2
$ docker images
REPOSITORY                     TAG                 IMAGE ID            CREATED             SIZE
indigodatacloud/orchestrator   DEEP-2            xxxxxxxxxxx        1 hours ago        843MB

Documentation

Please find DEEP-2 services and components documentation at Included components.

Support

Most complex software contains bugs, we are not an exception. One of the features of free and open source software is the ability to report bugs, helping to fix or improve the software you use.

DEEP - HybridDataCloud project uses the INDIGO Catch-All GGUS - Support Unit and the deep-support@listas.csic.eu for general support requests. More details regarding each product support channels are provided in the respective products release pages.

Developers, researchers and IT enthusiasts: feel free to write to deep-info@listas.csic.es to ask for more information on how to use DEEP solutions for your work. For automatic notifications you can register to the DEEP - HybridDataCloud release RSS feed or subscribe to the DEEP - HybridDataCloud Announce Mailing list. You can also socialize with us via Twitter or by joining our LinkedIn group.