I'm using Python to invoke another program in a sub-process. I've noticed my memory sometimes gets so large as to crash the system, and I'm wondering if I'm not correctly cleaning up the memory somehow.

Here's what I have:

children = []

fruits = ["apple", "banana", "cherry"]

for fruit in fruits:
	children.append(subprocess.Popen(["fruitProgram", fruit]))

while len(children) > 0:
	child = children.pop()
	child.wait()

subprocess.run("doStuffWithFruitProgramOutput")

I've had my doStuffWithFruitProgramOutput crash due to lack of memory when run like this, but not when run by itself. Is child.wait() not enough to free the child process resources?

I tried adding gc.collect() after clearing children, and that doesn't seem to have helped.

Is there something else I need to be doing? Or am I barking up the wrong tree here, and my problem is somewhere else?

posted over 3 years ago

CC BY-SA 4.0

3y ago by Patol75‭

Hyperlynx‭

285 reputation 7 5 35 10