This repository contains a Python module and an R package. The purpose of both is to process and analyse the database contents of OmniPath, export tables and create figures. As the data always comes from the pypath Python module, the omnipath2 Python module in this repository is responsible for the extraction of the data from all main database domains of OmniPath (network, enzyme-substrate, complexes, annotations, inter-cellular communication).

The omnipath2.database module builds all databases according to the parameters provided and saves them into pickle files. This way to load all databases takes only 2-5 minutes.

Various modules extract information from the databases and generate tables or create figures. For almost all plotting we use the matplotlib powered base class in omnipath2.plot.

Some of the tables are supplementary tables, but we compile a number of tables to provide input for the R package.

The omnipath2 R package in the R directory reads these tables, further processes the data and creates further figures.

The main module contains a description of the workflow and a class to call all its elements.

The class database.Database has the methods to compile the data in order to make sure we use the databases always the same way. Also it saves each database to pickles and serves them to all the other modules. It is instantiated in __init__.py hence you can do for example to get the omnipath network:

import omnipath2
op = omnipath2.data.get_db('omnipath')

At the moment I created 2 PPI networks, one is called omnipath and corresponds more or less to the data in the web service, it contains all the extra datasets; and the other one called curated and contains only the literature curated data. In addition we have the TF-target, miRNA-mRNA and TF-miRNA networks.

A number of methods for combining the network with the intercell annotations are already in the pypath.annot.CustomAnnotation class, hence we don't need to implement these in omnipath2. Also for each database domains lots of methods to query various numbers and subsets are in the relevant class in pypath, so we only need to call these. Just 2 examples: pypath.intercell.IntercellAnnotation.degree_inter_class_network_inhibitory will provide the degree distribution of inhibitory edges between 2 intercell classes, or pypath.main.PyPath.interactions_mutual_by_resource will return the mutual interactions grouped by resource, and so on.

The r_preprocess module exports tables for the R plotting scripts -- By default figures and tables are exported to the figures and tables directories into a subdirectory with the current date e.g. tables/20191028/intercell_classes_20191028.tsv, also each file name by default contains the date. In the R directory there is an R package called omnipath2; this works from the tables exported by the Python module.

The supptables module compiles and exports the supplementary tables S2-S6 with lots of numbers about each database domain.

The file names are in the omnipath2.settings module and might contain variable fields as for certain figures we have more different versions, e.g. compiled from different networks

The omnipath2.colors module reads the palettes and provides them to the other modules.

The omnipath2.plot module contains a base class PlotBase which we use for most of the plots. The omnipath2.intercell_plots, omnipath2.annotation_plots, omnipath2.network_plots, omnipath2.complexes_plots modules have the plots created by Nico and wrapped into classes inheriting from omnipath2.plot.PlotBase.

The directories can be customized in the settings module. One directory, by default pickles, contains the pickle dumps of the databases. The figures and tables directories contain the figures and tables generated by the module. Both the Python and the R methods use the same directories. Optionally the figures and tables can be collected into timestamped subdirectories which helps to keep track of their development, check or remove the old ones or find the most recent ones. Also optionally a timestamp can be added to each file name. The timestamps by default are an 8 digit representation of the date, so every day a new directory will be started (except if you keep running the same session over midnight, then still the old directory will be used). The files.json file keeps track of the most recent versions of all tables and figures generated both for the Python and R part as well as the earlier versions.

With all databases loaded Python requires maybe around 5G memory. The inter-cellular network data frame requires little more than 1G. pandas operations can result 1-3G peaks on top of that baseline. Overall it's good to have 8G RAM available.

You can run the whole workflow by calling main.py.

python main.py

You can run selected parts or tasks only:

from omnipath2 import main

# 2 parts selected (each a series of tasks)
workflow = main.Main(parts = ['intercell_plots', 'network_plots'])
workflow.main()

# only selected tasks:
workflow = main.Main(steps = main.workflow['intercell_plots'][0])
workflow.main()

The R workflow is called by the r_plotting part of the Python one, but also can be run separately:

Rscript omnipath2_workflow.r