- Table of Contents
- HPC connection (will be added soon)
- Room for modification
pyiron is an integrated development environment (IDE) for computational materials science. It integrates several computational/analysis tools in a common platform and provides various functionalities for creation, management and sharing of your workflows.
In order to run pyiron, prepare for the following hardware-requirements:
- for running jupyterhub: > 2 GB of RAM, 2 VCPU
- overall resources on the server should be proportional to the number of users: ~2 GB of RAM, 2 VCPU, 10 GB of storage per user
Morever, we assume the following:
- This setup uses Keycloak as identity provider. Therefore, a client id and secret are needed.
- For simplicity, we assume that all jobs are run on the same server as the jupyterhub container.
- The configuration of the hostnames is assumed to be done separately by the admin, e.g. creating the A-record, etc
Clone this GitHub repository into your pmd-server’s root directory:
1 git clone https://github.com/materialdigital/pyiron_workflow_environment_deployment pyiron
Just to give you a brief overview what’s in there:
||a json file containing the configurable environment variables: keycloak client details, name of pyiron docker images, users’ resources, and Postgres database Password|
||a json file containing the non-configurable environment variables|
||A template of the configuration file for the Nginx reverse proxy|
||the compose file, which runs jupyterhub and its Postgres database services.|
For the Nginx reverse proxy, you need to provide a configuration file. A template is provided in the repository under
After adding the domain, and the path to the SSL certificates and key (as described here), you should copy it to the nginx directory:
1 cp nginx/pyiron.conf ../data/nginx/
Of course, here it is assumed that the PMD-S core repo is the parent directory of the pyiron deployment directory.
Submit a request for pyiron SSO credentials via the Material Digital contact form. In the form provide the URL you have reserved for your pyiron instance.
If you choose to use a local Keycloak instance, you can create a keycloak-client via the following steps:
1) Sign in as the admin of the keycloak instance
2) Select your desirable realm
3) Select clients from the left panel
4) On the top right-hand side, select create
5) Enter a client ID, and a root URL. For the URL, you should provide the url of pyiron
6) click save
7) In the setting tab, change the access type to confidential, and save
8) From the credential tab, you can obtain the client secret
In the next step, You need to customize the central configuration file
config.json. There, you will need to provide tag-names for a set of pyiron
docker images you can choose from when starting a session.
Pyiron offers various docker images corresponding to its modules: atomistics,
continuum, md, etc.
config.json to reflect your environment: be sure to replace the
<pyiron_domain> in the URLs with the
correct destination for the keycloak-instace (either local or shared across
PMD-participants) and for the pyiron-service at hand.
Moreover, you need the keycloak-realm (placeholder
<realm>, the client id, and
the client secret (see Step 3. if you chose to configure keycloak by yourself).
Also, do not forget to choose eligible image-tags for base, atomistic, continuum and experimental (see Step 4).
The details of the keys in
config.json can be found in the table below.
||Keycloak Token URL; here you need only to change the domain and the realm”|
||Keycloak authorize URL; here you need only to change the domain and the realm|
||Keycloak userdate URL; here you need only to change the domain and the realm|
||Keycloak call back URL; here you need only to change the domain|
||The client ID defined in Keycloak|
||The secret for the client, provided from the Keycloak instance|
||The relevant information of pyiron_base image in the form of
||The relevant information of pyiron_atomistics image in the form of
||The relevant information of pyiron_continuum image in the form of
||The relevant information of pyiron_experimental image in the form of
||The limiting amount of RAM per user|
||The limiting amount of VCPU per user|
||The username of jupyterhub admin, this username should be consistent with the username in the keycloak instance|
||A password for the postgres database|
To generate the environment files, run the
configure.py script from the parent directory. The script can also be found here
1 2 3 4 5 6 cd pyiron # run `configure.py` script from the pmd-server parent directory # in the case that python3 is installed on the host OS: python3 ../scripts/configure.py # otherwise you can use a docker container to run the script via: docker run --rm -v $PWD/:/tmp/ -v $PWD/../scripts/configure.py:/tmp/configure.py-w /tmp python:3-alpine python configure.py
Run the compose-script to start the containers via:
1 docker-compose up -d
In principal, the pyiron docker containers can submit jobs to the cluster according to pyiron documentation in here. This feature will be added to pyiron docker images in the next release.
Here we assumed a semi-automated deployment with minimal changes needed from the side of IT administrators. However, many things can be changed, such as:
- building a customized jupyterhub; this gives the possibility to change the jupyterhub configuration
- In the current setup, we assumed to have four jupyter environments based on:
pyiron_experimental. This list can be extended in the case of a customized jupyterhub build.