[tutorial_ml:02] Python project setup, and installation
Section 002
Files and configuration
In the structure of our Python project, three components stand out as pivotal: src, setup.cfg, and pyproject.toml. Each plays a critical role in organizing, configuring, and managing the package.
> src
The src directory is the heart of our package, encapsulating all the functional code. This dedicated source directory offers several benefits:
- Centralization: By keeping all package-related code in one place,
srcsimplifies navigation and management of the project's contents. - Isolation: Housing the source code in
srchelps in isolating it from other project files, like tests and documentation. - Clarity and Maintenance: A clear separation of source files enhances the readability and maintainability of the project.
> setup.cfg
The setup.cfg file is a configuration file for setuptools, a library used to facilitate packaging Python projects. This file allows you to declare various metadata and options related to your package, which are used when building, distributing, or installing it. Here's how it's utilized in your project:
- Metadata: It includes information like the package name, version, author details, and description. This metadata is essential when you publish your package to repositories like PyPI.
- Package Options: You can specify which directories contain your package’s source code, additional files to include, dependencies, and more.
- Customization: Through
setup.cfg, you have the flexibility to customize various aspects of your package's build and distribution process, such as specifying entry points for creating command-line tools.
In essence, setup.cfg makes your package configuration more organized and readable, separating the configuration from the script logic typically found in setup.py.
[metadata]
name = <your-package-name>
version = <your-package-version>
author = <your-name>
author_email = <your-email>
description = <your-description>
long_description = file: README.md
long_description_content_type = text/markdown
url = https://github.com/<your-user>/<your-project>
project_urls =
Bug Tracker = https://github.com/<your-user>/<your-project>/issues
classifiers =
Programming Language :: Python :: 3
License :: OSI Approved :: MIT License
Operating System :: OS Independent[options]
package_dir =
= src
packages = find:
python_requires = >=3.6, <4install_requires= # add here all the python packages that are required
numpy==1.26.2
scikit-learn==1.3.2
pydantic==2.5.1[options.packages.find] # tells where to look for your code
where=src[options.package_data]
* = *.md
> pyproject.toml
pyproject.toml modernizes and simplifies the way Python projects are configured and built. Its adoption signifies a move towards more standardized and efficient Python project management, benefiting developers and users alike by providing a clear, consistent, and comprehensive configuration file.
[build-system]
requires = ["setuptools", "wheel"]In our current setup, we’ve implemented a straightforward build system configuration. However, future articles will delve into more sophisticated and robust build-system setups. These enhancements will be tailored to maximize the efficiency and functionality of our applications, addressing more complex scenarios and requirements.
Development environment
pip install -e .
Editable Mode: When you install a package using pip install -e ., pip doesn’t copy files to the site-packages directory. Instead, it creates a link (a .egg-link file in the site-packages directory) to your project directory. This means any changes you make to the source code are immediately reflected in the installed package, without the need for re installation.
Real-Time Changes: Because the installed package is a direct reference to your project files, any modifications you make to the code base are instantly active. This is incredibly useful for development and testing, as you can immediately see how your changes affect the package’s behaviour.
Simplifying the Workflow: This approach eliminates the need to build and reinstall your package every time you make changes. It streamlines the development workflow, making it more efficient and less error-prone.
Testing and Debugging: pip install -e . is particularly useful for testing. You can write and run tests against the latest version of your code without going through the build-install-test cycle. This immediate feedback loop is invaluable for rapid development and debugging.
Integration with Source Control: Since your package is installed from your local development directory, it’s easy to keep it under version control (like Git). This integration ensures that your development environment stays consistent with your source code repository.
Dependencies Handling: When you use pip install -e ., pip also installs any dependencies specified in your setup.py or setup.cfg file. This ensures that your development environment closely matches the environment where your package will eventually be deployed.
Usage Example
Assuming you’re in the root directory of your Python package (where setup.cfg is located), you would run:
pip install -e .
#This command makes your package available in your current Python environment, and any changes made to the source code will be immediately reflected.Deployment
Step 1: Install build Install the build module, ideally in a virtual environment:
pip install buildStep 2: Navigate to your package’s root directory and run:
python -m buildThis command creates two distributions in the dist/ directory: a source archive (.tar.gz) and a wheel file (.whl).
Step 3: Installing Your Package Install your package using pip:
pip install dist/your_package_name-version-py3-none-any.whlRemember to replace the filename with the actual name of your wheel file.
Additional Notes: The build module is a straightforward tool for building Python packages, aligning with PEP 517.
Previous Section 001: Project Structure and Functionalities
Next Section 003: Testing and Experimenting with your package
Code base
Access the entire code on github.com/rfelixmg/tutorial_ml
Remember to fork and star if you like it!
