How do you manage efficiently multiples jobs with modern computer cores ? This is a real problem that had me stuck for a while. I am working on a framework to analyze next generation sequencing data, consequently I have numerous files to manages, differents types of data that have to undergo a specific analysis. On the other hand, I have a few cores (24), some RAM (32Go) and decent hard drives. Read more

The analysis of sequencing data (at least in the first steps) is often a matter of starting multiple scripts one after an other. I found out that to log the stdout and the stderr with python was not that much obivous for me. I struggled quite a bit with the logging module but finally I came with the following. import os import subprocess import logging import logging.handlers import datetime as dt class ShellLaunchAndLog: ''' logging of shell commands output ''' def __init__(self): self. Read more