Oneprovider deployment — practice agenda

  • get familiar with the Onedata@FR environment, assign roles,
  • prepare the VM with an ansible playbook,
  • create a space in Onezone,
  • run the Onedatify deployment wizard (CLI),
  • test the deployed Oneprovider.

Onedata@FR

In this chapter, we will start building an environment simulating a Onedata zone, bringing together different data providers in France.

We will use demo.onedata.org as the Onezone instance, to which all providers will be registered.

Everybody will be assigned a single Oneprovider to deploy and maintain, and a single distributed space to administer (the assignee should coordinate the space setup according to the Onedata@FR outline).

The Onedata@FR environment will be used for practice throughout the remaining workshop chapters.

Onedata@FR

Get familiar with the Onedata@FR outline — find it here.

no-margin centered

The host assigns the French cities to participants.

Please put down your name as the owner in the spreadsheet.

Prepare the VM (1/4)

  1. SSH to your virtual machine where Oneprovider will be deployed.

  2. Clone the onedata-deployments repo:

    git clone https://github.com/onedata/onedata-deployments.git

  3. Enter the virtual machine's initial deployment directory:

    cd onedata-deployments/initial-vm-config/ansible/

    Side note: below steps are documented in the README.md.

  4. Install ansible and other prerequisites:

    sudo apt update
sudo apt install -y python3 python3-pip
sudo python3 -m pip install ansible "Jinja2>=2.10,<3.1" jmespath

Prepare the VM (2/4)

  1. Make sure that an additional empty volume is attached to the VM (it will be used for an LVM volume).

    sudo lsblk

    Identify the volume (e.g. /dev/sdd), it will be needed in the next steps

  2. Edit the hosts file (ansible inventory file), and put down localhost:

    [myhosts]
# node names (node, node1 ...) will be used as the first element of the node's FQDN - see group_vars/all.yaml.

# run for a single node
node ansible_host=localhost

Prepare the VM (3/4)

  1. Edit the group_vars/all.yml file. Put down the proper domain, subdomain (as assigned in the Onedata@FR outline), and disk_dev. The vg_name can be the default.

    domain: demo.onedata.org   # domain name
subdomain: ???             # subdomain name
                           # The FQDN of the node will be:
                           #   {{inventory_hostname}}.{{subdomain}}.{{domain}}
                           # inventory_hostname is the node name from the hosts file

disk_dev: /dev/???   # path to the disk dev for an LVM volume. The existing data will be lost!
vg_name: onedata-vg  # LVM volume group name

  2. Configure SSH connection to the localhost:

    [ ! -f ~/.ssh/id_rsa ] && ssh-keygen -b 2048 -t rsa -f ~/.ssh/id_rsa -q -N ""  # if not generated yet
cat ~/.ssh/id_rsa.pub >> ~/.ssh/authorized_keys

    not needed if you are using ssh-agent.

Prepare the VM (4/4)

  1. Run the playbook:

    ansible-playbook -i hosts site.yml

    There should be no errors — let us know if there are.

  2. The machine is ready for Oneprovider deployment. In upcoming chapters, we will take a closer look into this ansible playbook.

Onedatify

Onedatify is a CLI wizard for Oneprovider deployment, currently in beta.

It has some loose ends and imperfections but has proven itself as a very quick, convenient, and dependable way of deploying the Oneprovider service.

As far as dockerized deployments are concerned, we strongly suggest using docker-compose, which helps keep things clean. Onedatify builds upon docker-compose, collecting user input during the installation and building the proper config.

Obtain the Onedatify deployment oneliner

  1. Log in to demo.onedata.org.
  2. Create a space called test (we aren't creating the Onedata@FR spaces yet).
  3. Find the Add support button in the overview or in the Providers tab of the space.
  4. Switch to Deploy your own Oneprovider tab and copy the oneliner command.

centered

Run the oneliner

  1. SSH to your oneprovider VM, paste the oneliner in the terminal, and hit <ENTER>.
  2. Follow the wizard, answering the questions as follows:
    • display name — use the name from the spreadsheet,
    • emergency passphrase — leave blank for autogenerated one,
    • email address — put down yours,
    • absolute path to persistence — leave blank for the default,
    • latitude, longitude — use the values from the spreadsheet,
    • use subdomain delegation — y,
    • subdomain — use the subdomain from the spreadsheet,
    • what storage — posix,
    • directory for the space data — leave blank for the default /tmp (this space is just for a test),
    • support size — 1GiB.
  3. Wait for the script to end.
  4. If any error has occurred — let us know.

Test via Web GUI

Go to the test space in the GUI, try to upload, and then download some files.

Alternatives to Onedatify

  • Manually maintained docker-compose + batch deployment

    ONEPROVIDER_CONFIG section of the compose file — covered in the next chapter.

  • Manually maintained docker-compose + GUI deployment

    The service can be deployed using a graphical wizard, guiding you through the steps similarly to Onedatify.

  • Helm charts — Kubernetes deployment.

Space privileges regarding support

 


Privilege Actions
Add support add support to space
Remove support remove support from space

Cluster management privileges

 

Privilege Actions
View cluster view cluster members and memberships
Modify cluster edit cluster details
View privileges view cluster members privileges
Set privileges edit cluster members privileges
Remove cluster remove cluster
... manage cluster's memberships in regard to other resources

Next chapter:

Oneprovider configuration and tuning — practice