Torrent Crawler

Streamlined torrent search and download for movies.

View changes · Request feature

Table of Contents

• About the Project
  • Build With
• Getting Started
  • Prerequisites
    • Docker
    • Make
    • Poetry (Optional)
  • Installation
• Usage
  • Environment Variables
• Roadmap
• Contributing
  • Contributors
• License
• Contact
• Acknowledgements

About the Project

This project is a command-line search engine and crawler designed to automate searching for torrent files on 1337x.to. It lets you quickly find torrents for movies, TV shows, and other content by simply providing a search title, saving you time browsing unreliable streaming websites.

Build With

Getting Started

Follow these steps to set up and run the project locally.

Prerequisites

Before running this project, ensure you have the following tools installed.

Docker

Docker is required to run the project in a containerized environment. It enables compatibility across platforms, including Windows.

• On Ubuntu, install Docker with the following commands:

  sudo apt update
  sudo apt install -y ca-certificates curl gnupg
  sudo install -m 0755 -d /etc/apt/keyrings
  curl -fsSL https://download.docker.com/linux/ubuntu/gpg | sudo gpg --dearmor -o /etc/apt/keyrings/docker.gpg
  echo \
    "deb [arch=$(dpkg --print-architecture) signed-by=/etc/apt/keyrings/docker.gpg] https://download.docker.com/linux/ubuntu \
    $(lsb_release -cs) stable" | sudo tee /etc/apt/sources.list.d/docker.list > /dev/null
  sudo apt update
  sudo apt install -y docker-ce docker-ce-cli containerd.io docker-buildx-plugin docker-compose-plugin

• On Windows, download and install Docker Desktop from the official Docker website.

Make

Make is used to simplify command execution through the provided Makefile. You can also run the commands manually if preferred.

• On Linux, install Make with:

  sudo apt update
  sudo apt install -y make

• On Windows, you can use GnuWin, or run Make through WSL (Windows Subsystem for Linux).

Poetry (Optional)

Poetry is used for dependency management if you choose to run the program locally outside of Docker. This is supported on Linux systems only.

curl -sSL https://install.python-poetry.org | python3 -

After installation, ensure Poetry is added to your PATH:

export PATH="$HOME/.local/bin:$PATH"

Note

Poetry is not required when using Docker, as all dependencies are already included in the image.

Installation

Follow these steps to set up the project on your local machine.

1. Clone the project.

git clone https://github.com/Meleagrista/py-torrent-crawler.git

2. Go to the project directory.

cd py-torrent-crawler

3. Make sure Docker is installed and the daemon is active.
4. Build the project using Make.

make build

Usage

Follow the steps below to get started and use the project effectively.

make run

You can modify the parameters DOWNLOAD_DIR and CACHE_DIR in the Makefile to set the download and cache directories, respectively to your desired locations. You can also change them at runtime by running the command with the following parameters:

make run DOWNLOAD_DIR=/path/to/download CACHE_DIR=/path/to/cache

Environment Variables

You can customize the behavior of the program by setting the following environment variables:

• TORRENT_FILES_PER_MOVIE Specifies the default number of torrent files to download per movie.
• TORRENT_SUPPORTED_LANGUAGES Defines the languages that are supported for torrent file downloads.
• SELENIUM_LOAD_STRATEGY Configures the loading strategy for Selenium, which is only relevant if the source website is blocked on your network.
  • The recommended option is normal or you can choose eager for faster loading times with increased risk of errors.
• TERMINAL_WIDTH Sets the width of the terminal output. Adjust this value to match your terminal size for optimal display.

Roadmap

The following features and enhancements are planned for development.

• Add screenshots and a demo to documentation.
• Add subtitle search by extracting metadata from torrent files.
• Add alternative torrent sites for scrapping.
• Improve search functionality to support other media types.
  • Add support for TV shows and series.

Contributing

If you have a suggestion that would make this better, please fork the repo and create a pull request. You can also simply open an issue with the tag "enhancement". Any contributions you make are greatly appreciated.

1. Fork the project
2. Create your feature branch.

git checkout -b feature/AmazingFeature

3. Commit your changes.

git commit -m 'Add some AmazingFeature'

4. Push to the branch.

git push origin feature/AmazingFeature

5. Open a pull request

Contributors

License

This project is licensed under the GNU General Public License v3.0. See the LICENSE.txt file for full license details.

Contact

Feel free to reach out if you have any questions or feedback.

Martín do Río Rico - mdoriorico@gmail.com

Acknowledgements

Special thanks to the following projects and resources that inspired or supported this project:

• TorrentP – A lightweight torrent download tool that served as inspiration.
• libtorrent – The powerful BitTorrent library.
• Choose an Open Source License - For helping to select the right license for this project.