LogFAQs > #955701089

1337toothbrush posted...

Within Python there are limitations for squeezing performance, though. If you wanted to take advantage of multiple threads/cores, you'll run into the GIL (Global Interpreter Lock). Only one thread can control the Python interpreter at a time. The "solution" is to use multiple processes instead of multiple threads, but then you have to pass data between processes. The thing is that there are entire classes of multiprocessing where you need to deal with a lot of data, so passing around huge amounts of data can destroy performance gains you get from splitting the workload.

Also, various aspects of Python's design prevents optimizations from the interpreter. There are a whole bunch of issues with optimizing within Python, that's why the most performant libraries are written in other languages like C or C++ and then compiled to be called from Python.

yea my point was that the code is probably written incompetently at a really high level for most people learning

e.g. manually looping through rows in a big pandas df instead of using vectors, or dumb things with strings
---

These old bones live to learn her wanton ways:
(I measure time by how a body sways).