Final Project - GitHub Folder Structure

Overview

This structure keeps your work organized and easy for us to run. It is the recommended layout for your Midterm submission.

  • Track A: you may do the analysis in notebooks. Every notebook must run after Restart & Run All.
  • Track B: repeated steps must be written as functions, either in a .py module in analysis/ or in clearly separated notebook sections, and your repo must include a README and a requirements.txt.

Both tracks: use relative paths only, and include both raw and processed data.

Folder Structure 

your-project/
├── data/
│   ├── raw/
│   └── processed/
├── visualizations/          # .png and/or .html (≥1 map required)
├── notebooks/               # Track A and/or B
├── analysis/                # Optional / Track B: .py files with functions, classes, etc.
├── requirements.txt         # Track B: packages your analysis uses (uv.lock if you are using uv)
├── README.md                # Track B: how to run your analysis and where to get the data
├── .gitignore               # Files to exclude from version control (GitHub only)
└── Group_<group_number>_midterm.pdf

How to Use Each Folder

data/ – Datasets

  • raw/ → Original, unmodified datasets. Never edit these files directly.
  • processed/ → Cleaned data produced by your code. Your code must be able to regenerate these files from raw/.

If your data is larger than 98 MB, do not commit it to GitHub; include it in the ZIP you submit.

visualizations/ – Charts and Maps

Save every figure your report uses here, as .png and/or .html. The midterm needs at least 1 map and 3–5 charts; static images are fine at this stage.

notebooks/ – Notebooks

  • Track A: your analysis can live here. Each notebook must run top to bottom after Restart & Run All, without errors.
  • Track B: notebooks are fine as long as repeated steps are written as functions, either in clearly separated notebook sections or in a .py module in analysis/. This is a midterm requirement for Track B.

analysis/ – Python Scripts (Track B, or optional for Track A)

Put your .py files here. Write repeated steps as functions instead of copying code, so the same cleaning or plotting step is defined once and reused.

Example files (suggestions only):

  • data_processing.py – all of your data cleaning steps, runnable from top to bottom
  • visualize.py – functions for generating charts and maps

Making analysis/ importable with __init__.py

If you keep functions in a .py module here and want to call them from a notebook, add an empty file named __init__.py to the folder. Python then treats analysis/ as a package, and every .py file in it becomes a module you can import by name.

analysis/
├── __init__.py        # empty file; makes analysis/ a package
├── data_processing.py # def clean_data(df): ...
└── visualize.py       # def create_map(gdf): ...

Create it from the project root:

touch analysis/__init__.py

Then, from a notebook in notebooks/, put the project root on the path and import through the package name:

import sys
sys.path.append("..")
from analysis.data_processing import clean_data

Report

Put the report PDF at the project root, for example Group_<group_number>_midterm.pdf.

How to Run Your Code

Your data cleaning steps must run from top to bottom without errors from one notebook or script (e.g. data_processing.py or data_processing.ipynb). That file may import functions from your other .py files in analysis/.

Notebooks. Open it and run Kernel → Restart & Run All.

Scripts. From the project root:

python analysis/data_processing.py

Track B: your README.md must explain how to run your analysis and where someone else would get your data, whether that is a download script, an API call, or a note about any manual steps needed. Track A groups are encouraged to include one too.

Installing Python Packages

Track B must include a requirements.txt; Track A groups are encouraged to. List every package your project needs, one per line, and pin a version if your code depends on it:

pandas==2.1.0
geopandas
matplotlib

Install them from the project root:

pip install -r requirements.txt

If you are using uv, commit uv.lock as well. If you are working in Colab only, add a top cell: !pip install -r requirements.txt

Keep the list minimal, and update it whenever you add a dependency. See Reproducibility for more on environments.

Advanced Users

For more advanced teams, this is a preferred and recommended folder structure:

your-project/
├── src/               # Main source code
│   ├── __init__.py    # Makes src a package
│   ├── main.py        # The main script to run the full analysis
│   ├── data/          # Code for loading and processing data
│   │   ├── __init__.py
│   │   ├── process_data.py  # Functions to clean and prepare data
│   │   ├── load_data.py     # Functions to read data from files
│   ├── analysis/      # Code for running analysis
│   │   ├── __init__.py
│   │   ├── compute_metrics.py  # Functions for statistical/model analysis
│   ├── visualization/ # Code for visualizations
│   │   ├── __init__.py
│   │   ├── plot_results.py  # Functions to generate plots
│   ├── utils/         # General helper functions
│   │   ├── __init__.py
│   │   ├── helpers.py  # Miscellaneous helper functions
│   │   ├── config.py   # Configuration variables (e.g., file paths)
├── notebooks/         # Jupyter notebooks for testing and prototyping
├── data/              # Datasets (keep raw data unchanged)
│   ├── raw/           # Original, unmodified data files
│   ├── processed/     # Cleaned and formatted data for analysis
├── visualizations/    # Charts and maps (.png and/or .html)
├── outputs/           # Other results (tables, model outputs)
├── requirements.txt   # List of required Python packages
├── README.md          # Instructions and project overview
├── .gitignore         # Files to exclude from version control
└── Group_<group_number>_midterm.pdf

Leverage Feature Branches and Pull Requests: It is highly recommended that each team member work on their own feature branch. Use pull requests for merging changes into the main branch and for code review. This minimizes merge conflicts and enhances code quality.