Post History

## Root cause

Ultimately, `ModuleNotFoundError` is a kind of `ImportError`. It occurs when Python code tries to use an absolute import, but can't find the named module. (Failed relative imports will only raise the base `ImportError`. This makes sense because in principle, an absolute import involves *searching* for the module, but for a relative import Python should already know exactly where to look, according to the package structure.)

## Understanding `sys.path`

In general, Python looks for absolute imports by searching within folders that are named in the `path` attribute of the `sys` standard library module (`sys.path`). User code can modify or reassign the list like any other list; and reassigning it *does* affect module lookup.

**However, this is rarely necessary and should always be carefully considered**. Popular, complex third-party libraries often span tens of thousands, or even hundreds of thousands of lines of code without needing to do this.

At startup, `sys.path` is initialized [according to clearly documented rules](https://docs.python.org/3/library/sys.html#sys.path). Rather than trying to modify `sys.path` from within the code before attempting an `import`, it's normally a better idea to set the `PYTHONPATH` environment variable.

Normally, when Python is asked to do an absolute import, it will check each folder listed in `sys.path` - the folder itself **only**, not searching in any subdirectories - for a `.py` file with the right name, or some other implementation (e.g. a compiled `.pyc` bytecode file contained within a `__pycache__` subdirectory; in Python 2.x, the `.pyc` files were expected to be side-by-side with the corresponding `.py` source code). It's possible to modify this search process, but that's beyond the scope of this answer. Some "built-in" standard library modules, such as `sys`, bypass this search process; the module is directly created within the Python implementation. (Among other things, this makes it possible to `import sys` without needing `sys.path` to exist already.)

(TODO: factor out a separate Q&A for this?)

## Troubleshooting

Always check for typos first, of course.

If you didn't write the code to be imported yourself, it's important to understand whether that code is part of the *standard library* (something that comes with Python) or is *third party* needs to be installed separately. Check the [Python standard library documentation](https://docs.python.org/3/library/index.html) to know what is included. Everything else must be installed separately.

Normally, third-party code is installed using Pip (the `pip` command-line program) - but if you set up the Python environment using Anaconda, you should use its tool (`conda`) instead.

Some packages have custom installation instructions; in these cases you will have to read their documentation first.

### When importing from the standard library

If you are trying to import a standard library module, outside of a very few special cases, the module can really only be *not found* due to a typo. Of course *other* things can indeed go wrong.

<details><summary>But, for completeness:</summary>

#### TKinter

This is part of the standard library, but in some cases it may not be included with a Python distribution. This especially happens on Linux and MacOS machines, for a copy of Python that came with the operating system. To get TKinter, you may need to use the *system* package manager (not Pip) to install it. However, it's often better to install a separate Python version. Every Python installer from https://www.python.org should include Tkinter (and configure everything else needed for it on your system) automatically, and the defaults for compiling from source on Linux should include it as well.

(TODO: separate Q&A for details on this problem)

#### `zlib`

Similarly, Python may be built in such a way that it doesn't include the `zlib` standard library module.

#### OS-specific modules

Certain modules may only be available depending on the operating system, because they are used to provide OS-level functionality. These are listed in the "MS Windows Specific Services" and "Unix Specific Services" sections of the standard library documentation.

#### Modules removed or renamed in Python 3

Several standard library modules were [renamed or reorganized](https://docs.python.org/3/whatsnew/3.0.html#library-changes) for Python 3. Some old guides may refer to the Python 2 names of standard library modules, or old code may need to use something else in order to update for Python 3.

<details><summary>Here is a comprehensive listing of the changes:</summary>

|Module removed or renamed|Workarounds|
|-|-|
|`_winreg`|renamed to `winreg`|
|`ConfigParser`|renamed to `configparser`|
|`copy_reg`|renamed to `copyreg`|
|`Queue`|renamed to `queue`|
|`SocketServer`|renamed to `socketserver`|
|`markupbase`|renamed to `_markupbase` - **n.b. this is not intended for public use**; it's an implementation detail for `html.parser`|
|`repr`|renamed to `reprlib`|
|`test.test_support`|renamed to `test.support`|
|`Tkinter` (and several others)|Use `tkinter`; the module was renamed to conform to standard naming conventions. There used to be many Tkinter "package contents" that were actually top-level modules; now `tkinter` is a proper package.|
|`anydbm`, `dbhash`, `dumbdbm`, `gdbm`, `whichdb`|Use the [`dbm` package](https://docs.python.org/3/library/dbm.html#module-dbm). These used to be separate modules (including `dbm` itself); now `dbm` is a package including all this functionality.|
|`HTMLParser`|Use `html.parser`; see the new [`html` package](https://docs.python.org/3/library/html.html#module-html)|
|`htmlentitydefs`|Use `html.entities`|
|`urllib2`|Use `urllib.error` and `urllib.request`; see the [urllib package](https://docs.python.org/3/library/urllib.html#module-urllib). Python 2 had separate `urllib` and `urllib2` modules; now `urllib` is a package that includes `urllib2` functionality (and more).|
|`urlparse`|Use `urllib.parse`|
|`robotparse`|Use `urllib.robotparser`|
|`httplib`|Use `http.client`; see the [`http` package](https://docs.python.org/3/library/http.html)|
|`BaseHTTPServer`, `CGIHTTPServer`, `SimpleHTTPServer`|Use `http.server`|
|`Cookie`|Use `http.cookies`|
|`cookielib`|Use `http.cookiejar`|
|`xmlrpclib`|Use `xmlrpc.client`; see the [`xmlrpc` package](https://docs.python.org/3/library/xmlrpc.html)|
|`DocXMLRPCServer`, `SimpleXMLRPCServer`|Use `xmlrpc.server`|
|`sets`|**Removed**; use the built-in `set` type instead. (This is extremely obscure; this library was added in 2.3, and the built-in `set` type was added in 2.4.)|
|`new`|**Removed**; use `types` instead. The API is different, so code will need to be changed.|
</details>

In most cases, the `2to3` tool should be able to fix these imports automatically.

#### Modules removed in Python 3.12

<details><summary>

The [What’s New In Python 3.12](https://docs.python.org/3/whatsnew/3.12.html) document mentions that a few standard library modules are no longer available:</summary>

|Module removed|Workarounds|
|-|-|
|`asynchat`|Use the new `asyncio` module instead; the API is different, so code will need to be adapted.|
|`asyncore`|Use the new `asyncio` module instead; the API is different, so code will need to be adapted.|
|`distutils`|Upgrade to use Setuptools instead; but keep in mind that it's encouraged to use `pyproject.toml` as much as possible for configuration. Running the `setup.py` script directly is now deprecated; instead, Setuptools will invoke it for you during the building process.|
|`imp`|Use the new `importlib` module instead; the API is different, so code will need to be adapted. [Some basic guidance is included](https://docs.python.org/3/whatsnew/3.12.html#imp) in the documentation.|
|`smtpd`|The documentation recommends [the third-party `aiosmtpd` package](https://pypi.org/project/aiosmtpd/) instead.|
</details>

Many more old modules [will be removed in 3.13](https://docs.python.org/3/whatsnew/3.12.html#pending-removal-in-python-3-13), following [PEP 594](https://peps.python.org/pep-0594/).
</details>

### When importing from a third-party package

As well as checking for typos, make sure that you know *what name to import*. If you installed a third-party library from PyPI, the name that should be used with the `import` statement in your code *does not necessarily have anything to do with* the name used to find the package on PyPI. The PyPI name doesn't even have to be a valid Python identifier (in particular, it can use `-` characters).

<details><summary>Here are some commonly encountered mismatches:</summary>

|PyPI package name|Package names to use in code|
|-|-|
|`pillow`|`PIL` (`from PIL import Image` etc.)|
|`python-dateutil`|`dateutil`|
|`beautifulsoup4`|`bs4` (`from bs4 import BeautifulSoup`)|
|`opencv-python`|`cv2`|
|`google-api-python-client`|`googleapiclient`<br>(e.g. `from googleapiclient.discovery import build`). Some older versions may use `apiclient` instead.|
|`python-docx`<br>(There is also a `docx` PyPI package, but it is abandoned and does not support 3.x)|`docx`|
|`pywin32`<br>(previously `pypiwin32` and probably other names)|`win32api`, `win32com` etc. (many other top-level packages)|
|`pyserial`|`serial`|
</details>

If the code has the correct name and you have verified that the package has been installed (for example by using `pip list`, `pip show` with the PyPI package name, or trying to install again and getting a `Requirement already satisfied` message), **this happens because the script is using a different Python environment** rather than the one where the package was installed. (TODO: separate Q&A for this issue)

### When importing your own code

If you have code from other projects that you wrote yourself and need to use in the current project, generally you are encouraged to make an installable package and install it locally - even if you don't want to distribute it on PyPI (yet?). Failing that, however, you will need to make sure that `sys.path` includes the appropriate path for the other code.

However, normally, when you import "your own code" it will be something else that is part of the current project.

The **recommended way** to organize a Python project is to **have most of the code inside a package folder** (or sub-folders), perhaps with some simple "driver" scripts just outside. In normal cases, the search path will include the folder with the "driver" scripts, so they can easily do an absolute import of some specific module inside the package. **Then, within the package, use exclusively relative imports** to connect different parts of the project. This minimizes the dependence on the module search path and helps prepare for packaging the code properly for distribution.

That said, absolute imports usually don't cause this problem because the default `sys.path` (see the previous section on `sys.path`) should make it possible to find the other code. **The most common problem occurs when trying to use the code in interactive mode** (from the Python interpreter prompt). In this case, `sys.path` will normally include a `''` entry, which is a *relative* path to the current working directory. As a result, **using code like `os.chdir` can prevent modules from being found** in the expected place.