Development#

This guide covers setting up a development environment and contributing to read_aconity_layers.

Development Setup#

Prerequisites#

  • Rust 1.70+ with Cargo

  • Python 3.11+

  • Poetry

  • Git

Environment Setup#

  1. Clone and setup the repository:

    git clone https://github.com/Cian-H/read_aconity_layers.git
cd read_aconity_layers

  2. Install dependencies:

    poetry install --with dev,docs

  3. Setup pre-commit hooks:

    poetry run pre-commit install

  4. Build the Rust extension:

    poetry run maturin develop

Code Style and Quality#

This project uses several tools to maintain code quality:

Python Code#

  • Ruff: For linting and formatting

  • MyPy: For type checking

  • Pytest: For testing

Run quality checks:

# Format and lint Python code
poetry run ruff format .
poetry run ruff check .

# Type checking
poetry run mypy .

# Run tests
poetry run pytest

Rust Code#

  • rustfmt: For formatting

  • clippy: For linting

  • cargo test: For testing

Run quality checks:

# Format Rust code
cargo fmt

# Lint Rust code
cargo clippy

# Run Rust tests
cargo test

Testing#

The project includes comprehensive tests for both Python and Rust components.

Running Tests#

# Run all tests
poetry run pytest

# Run with coverage
poetry run pytest --cov=read_aconity_layers

# Run Rust tests
cargo test

Test Structure#

  • Python tests: Located in tests/ directory

  • Rust tests: Integrated into src/rust_fn/mod.rs

  • Property-based tests: Uses arbtest for Rust property testing

  • Regression tests: Validates against known good outputs

Adding Tests#

When adding new functionality:

  1. Add Rust tests in the appropriate module

  2. Add Python integration tests in tests/

  3. Update regression tests if output format changes

  4. Add property tests for mathematical functions

Documentation#

Building Documentation#

Prerequisites: You need the Rust toolchain installed for sphinxcontrib-rust to work.

# Install documentation dependencies
poetry install --with docs

# Build documentation
cd docs
make html

# Or build manually
poetry run sphinx-build -b html . _build/html

# Serve locally (optional)
make serve

Documentation Structure#

  • docs/conf.py: Sphinx configuration

  • docs/index.rst: Main documentation page

  • docs/python/: Python API documentation

  • docs/rust/: Rust API documentation

  • docs/*.rst: User guides and tutorials

The documentation automatically generates API references from:

  • Python docstrings and type hints

  • Rust documentation comments (/// and //!)

  • Type stub files (*.pyi)

Note: For Rust API documentation to work properly, you need:

  1. Rust toolchain installed (cargo, rustfmt)

  2. Proper Rust doc comments in your source code

  3. The sphinxcontrib-rust extension configured correctly

Contributing#

Workflow#

  1. Fork the repository on GitHub

  2. Create a feature branch from main

  3. Make your changes following the coding standards

  4. Add tests for new functionality

  5. Update documentation as needed

  6. Run the full test suite to ensure everything works

  7. Submit a pull request

Pre-commit Checks#

The project uses pre-commit hooks that run automatically:

  • Code formatting (Ruff, rustfmt)

  • Linting (Ruff, Clippy)

  • Type checking (MyPy)

  • Version bump validation

  • Poetry validation

These checks must pass before commits are accepted.

Release Process#

  1. Update version in Cargo.toml (triggers version validation)

  2. Update changelog if applicable

  3. Ensure all tests pass

  4. Create a release on GitHub

  5. CI automatically builds and publishes wheels to PyPI

Architecture Notes#

The library is structured in two main components:

Rust Core (src/rust_fn/)#

  • High-performance file I/O using CSV reader

  • Parallel processing with Rayon

  • Memory-efficient array operations with ndarray

  • Coordinate correction algorithms

Python Bindings (src/lib.rs)#

  • PyO3 integration for seamless Python interop

  • Error handling conversion from Rust to Python exceptions

  • NumPy integration for zero-copy array passing

  • Type annotations via stub files

Performance Considerations#

  • File I/O is the primary bottleneck

  • Parallel processing scales well with core count

  • Memory usage is proportional to dataset size

  • Coordinate corrections use vectorized operations

Common Development Tasks#

Adding a New Function#

  1. Implement in Rust (src/rust_fn/mod.rs)

  2. Add Python binding (src/lib.rs)

  3. Update type stubs (read_layers.pyi)

  4. Add tests for both Rust and Python

  5. Update documentation

Debugging Build Issues#

  • Check Rust version: Must be 1.70+

  • Verify PyO3 compatibility: Should match Python version

  • Clear build cache: cargo clean and poetry env remove --all

  • Check dependencies: Ensure all dev dependencies are installed

Profiling Performance#

For Rust code:

# Profile with perf (Linux)
cargo build --release
perf record --call-graph=dwarf ./target/release/your_binary
perf report

For Python integration:

# Profile with py-spy
pip install py-spy
py-spy record -o profile.svg -- python your_script.py