

See , How easy to download Files From URLs using Python
Below are the few possible ways of downloading files from URL's
1. Downloading Files Using requests module
The requests library is the most readable and widely used option.
✅ Basic Download
import requests
url = "https://example.com/sample.pdf"
response = requests.get(url)
if response.status_code == 200:
with open("sample.pdf", "wb") as file:
file.write(response.content)
print("File downloaded successfully!")
else:
print("Failed to download file")
✅ Streaming Large Files
import requests
url = "https://example.com/large_video.mp4"
response = requests.get(url, stream=True)
if response.status_code == 200:
with open("large_video.mp4", "wb") as file:
for chunk in response.iter_content(chunk_size=1024):
file.write(chunk) print("Large file downloaded successfully!")
2. Downloading Files Using urllib
Great for quick scripts and environments where installing packages is restricted.
✅ One‑Line Download
import urllib.request
url = "https://example.com/data.csv"
filename = "data.csv"
urllib.request.urlretrieve(url, filename)
print("File downloaded!")
3. Downloading Files Using wget
If you want a Python version of the Linux wget tool:
✅ Using wget
import wget url = "https://example.com/report.pdf"
wget.download(url, "report.pdf")
print("Downloaded!")
4. When to Use Which Method?
requests-> Most use cases,Clean API, supports streaming- But this Requires installation
urllib -> Quick scripts, restricted environmentsBuilt‑inLess user‑friendly
wget -> Simple downloads, resume supportVery easy to useExtra dependency
5.Parallel and Async Downloads are possible using the below 2 libraries
- ThreadPoolExecutor for parallel downloads
- aiohttp for async downloads
6.Parallel Downloads with ThreadPoolExecutor
This is perfect when:
- You’re downloading many small/medium files
- The bottleneck is network latency
- You want simple, readable code
✅ Fully Working Template
import requests
from concurrent.futures import ThreadPoolExecutor, as_completed
def download_file(url, filename):
try: response = requests.get(url, stream=True, timeout=10)
response.raise_for_status()
with open(filename, "wb") as f:
for chunk in response.iter_content(chunk_size=1024):
f.write(chunk)
return f"{filename} downloaded"
except Exception as e:
return f"Failed: {url} -> {e}"
urls = [ ("https://example.com/file1.pdf", "file1.pdf"),
("https://example.com/file2.pdf", "file2.pdf"),
("https://example.com/file3.pdf", "file3.pdf"), ]
with ThreadPoolExecutor(max_workers=5) as executor:
futures = [executor.submit(download_file, url, name)
for url, name in urls]
for future in as_completed(futures):
print(future.result())
⭐ Why this works well
- Uses threads (great for I/O‑bound tasks)
- Handles errors cleanly
- Streams chunks to avoid memory spikes
- Easy to scale by adjusting max_workers
7.Async Downloads with aiohttp (Super Fast)
Use this when:
- You’re downloading hundreds or thousands of files
- You want maximum concurrency
- You’re already in an async environment (or want speed similar to Node.js)
✅ Fully Working Async Template
import aiohttp
import asyncio
async def download_file(session, url, filename):
try:
async with session.get(url) as response:
response.raise_for_status()
with open(filename, "wb") as f:
async for chunk in response.content.iter_chunked(1024):
f.write(chunk)
print(f"{filename} downloaded")
except Exception as e:
print(f"Failed: {url} -> {e}")
async def main():
urls = [ ("https://example.com/file1.pdf", "file1.pdf"),
("https://example.com/file2.pdf", "file2.pdf"),
("https://example.com/file3.pdf", "file3.pdf"), ]
async with aiohttp.ClientSession() as session:
tasks = [ download_file(session, url, name)
for url, name in urls ]
await asyncio.gather(*tasks)
Calling is -> asyncio.run(main())
⭐ Why async is so fast
- No threads
- No blocking
- Thousands of concurrent downloads
- Ideal for large scraping pipelines
8.Based on speed
Module -> Best for -> speed-> complexity
--------------------------------------------------------------------------------------
ThreadPoolExecutor -> 10–200 files -> Medium -> Easy
aiohttp -> 200–10,000+ files->Very High->Medium
requests (single) ->1–10 files ->Low->Very Easy
Thanks,
