Setting up your system¶
The following is a guide to help you prepare your system for developing with Invenio. A proper development environment setup can save a lot of time and frustrations. Note that the following guide, is not meant as a guide for setting up servers.
We support development on macOS and Linux-based systems (development on Windows is not supported).
We recommend that you install libraries and tools using the Homebrew package manager. Homebrew allows you to install two types of packages - cask and normal.
Cask packages are usually UI applications that end up in your Applications folder. You can install cask packages using the command:
$ brew cask install <packages>
Here are some recommended cask packages (only
docker is required):
db-browser-for-sqlite # UI for SQLite discord # app that we use for chat among developers docker # docker setup for macOS google-chrome # needed for some end-to-end tests iterm2 # a better terminal than Terminal.app postico # UI for PostgreSQL spectacle # organise windows with keyboard shortcuts visual-studio-code # a text editor used by many Invenio developers
Normal packages are usually command line tools/libraries that end up in /usr/local and that you use from the CLI. You can install normal packages using the command:
$ brew install <packages>
The following packages are libraries that are likely to be needed during Invenio development in order to install certain Python packages:
cairo freetype geoip gettext glib imagemagick jpeg libffi libmemcached libpng libtiff libxml2 libxslt readline xz zlib
Following are CLI tools that are useful during development:
cookiecutter # tool to bootstrap new modules from templates chromedriver # selenium driver for chrome (used for end-to-end testing) gh # GitHub CLI client (useful e.g for checking out PRs) gifify # make short screen recordings for bug reports git # our version control system hub # extends git with github features
General CLI tools:
htop # a better top tree # pretty print a directory structure wget # http client zsh-completion # if you use zsh as shell base-completion # if you use bash as shell
CERN specific tools:
openshift-cli # if you deploy on openshift xrootd # library for accessing EOS storage cluster sshuttle # tunnel into CERN
nvm - both tools manage version of python and node
respectively. Install the following packages:
nvm pyenv pyenv-virtualenv pyenv-virtualenvwrapper
Once you have installed above packages, you can proceed with installing Python versions. The following will install Python 3.6, 3.7 and 3.8 and set the default Python installation to Python 3.8 (node you can always install the latest patch-level release):
$ pyenv install 3.6.9 $ pyenv install 3.7.8 $ pyenv install 3.8.5 $ pyenv global 3.8.5
You should edit your .bashrc or .zshrc file to initialise pyenv:
# nvm setup export NVM_DIR="$HOME/.nvm" [ -s "/usr/local/opt/nvm/nvm.sh" ] && . "/usr/local/opt/nvm/nvm.sh" # pyenv eval "$(pyenv init -)" # pyenv-virtualenv eval "$(pyenv virtualenv-init -)" # pyenv-virtualenvwrapper pyenv virtualenvwrapper
Now, you can create e.g. Python virtual environments using the following commands:
$ mkvirtualenv <name> $ mkvirtualenv -p python3.7 <name> $ workon <name>
In order to create e.g. DOI badges you need the DejaVu Sans font installed. Go to https://dejavu-fonts.github.io/ and follow the instructions.
Docker Desktop for Mac
You may need to increase the resources assigned to Docker Desktop for Mac See https://docs.docker.com/docker-for-mac/#resources.
A typical sign of needed more resources, is that services are not running or images are having problems building.
Want to write it? Contact us on the chat!
You can use any code editor of your choice. Here we give a brief overview of some of the editors our existing developers are using. For all editors, the most important is support for EditorConfig
All repositories have a
.editorconfig file which defines indention style,
text encoding, newlines etc. Many editors either come with built-in support
or plugins that reads the
.editorconfig file and configures your editor
Visit EditorConfig to see the list of supported editors.
Following editors are used by our existing developers. Don’t hesitate to reach out on our Discord server, to ask for help for useful plugins:
Working with Git and GitHub¶
There are a couple of utilities that allow you to work more efficiently with Git and GitHub.
GitHub CLI: command-line tool to interact with GitHub. Note that this tool is independent from the git CLI.
Hub: command-line wrapper around git that makes it easier to work with GitHub by adding new commands.
Use aliases to type frequent commands more efficiently. Note that your shell might offer more git aliases too, for example Oh My Zsh has the git plugin which among other things it will come with a few pre-configured aliases.
There are several tools for debugging, we will list here the ones which are editor agnostic:
pytest: comes with builtin support for dropping into pdb.
ipdb: this is a terminal debugger with features tab completion, syntax highlighting among others. Note that the debugging session is opened as the output of the running program.
wdb: web debugger with a client server architecture. The debugging session is opened in a web browser. Suitable for remote debugging and programs which do not allow to open an interactive shell session (i.e. Celery).