Run all jobs inside long lived worker processes
- There is less forking, once per worker process, instead of once per job. - Nice and ionice are applied to the worker processes and hence all jobs run by them. - This allows jobs to benefit from caches filled by previous jobs. For example many tools applied to a python file need to calculate if its python2 or python3.
This commit is contained in:
parent
39de82fb45
commit
667dd8ffa5
3 changed files with 47 additions and 40 deletions
|
|
@ -38,17 +38,6 @@ class LruCacheWithEvictionTestCase(unittest.TestCase):
|
|||
self._assert_cache(a, 1, 2, 2)
|
||||
|
||||
|
||||
class MultiprocessingWrapperTestCase(unittest.TestCase):
|
||||
|
||||
def test_multiprocessing_wrapper(self):
|
||||
def a(b, c, d=1):
|
||||
return b + c + d
|
||||
process = vigil.multiprocessing_process(a, 1, 2)
|
||||
result = process.result_conn.recv()
|
||||
process.join()
|
||||
self.assertEqual(result, 4)
|
||||
|
||||
|
||||
_DIMENSIONS = (40, 40)
|
||||
|
||||
|
||||
|
|
|
|||
Loading…
Add table
Add a link
Reference in a new issue