May 27, 2015

Run jobs in parallel with python

How do you manage efficiently multiples jobs with modern computer cores ? This is a real problem that had me stuck for a while. I am working on a framework to analyze next generation sequencing data, consequently I have numerous files to manages, differents types of data that have to undergo a specific analysis. On the other hand, I have a few cores (24), some RAM (32Go) and decent hard drives. Let’s combine everything together so that many files can be analyzed on multiple cores at the same time. Of note, I am mostly working with bacterial genomes, so my computing times in this respect might seem light with compared to human genome.. A good example (not too simple, not too complex ?) would be, how do I convert 10 sam files to 10 bam files IN PARALLEL ? Read more

Copyright 2025 - Mikael Koutero. All rights reserved.

Privacy Statement