Python中如何实现在线视频观看达到多线程下载的速度

Video Funnel - 让你在线看视频也能达到多线程下载的速度

马上使用：

  1. 从 PyPI 安装：
$ pip(3) install --user video_funnel
# or
$ sudo pip(3) install video_funnel
  1. 启动 video_funnel 的服务器：
$ vf http://tulip.ink/test.mp4
======== Running on http://0.0.0.0:8080 ========
(Press CTRL+C to quit)
  1. 用 mpv 播放：
$ mpv http://localhost:8080

动机：

众所周知，百度网盘之类产品的视频在线播放非常模糊，下载吧又限速，于是我写了 aiodl 这个下载器，通过 EX-百度云盘 获取的直链来“多线程”下载。可是每次都要下载完才能看又十分不爽，直接用 mpv 之类的播放器播放直链又因为限速的原因根本没法看，遂有了本项目。

实现思路：

  1. 先将视频按照一定大小分块。块的大小根据视频的清晰度而异，以下载完一个块后视频可以播放为准。可通过命令行参数 --block-size/-b 来指定，默认为 8MB。
  2. 对于上一步中的一个块，再次分块——为区别改叫切片，启动多个协程来下载这些切片，以实现“多线程”提速的目的。块和切片大小一起决定了有多少个连接在同时下载。切片的大小通过 --piece-size/-p 来指定，默认为 1MB。
  3. 一个块中的切片全部下载完后，就可以将数据传给播放器了。当播放器播放这一块的时候，回到第 2 步下载下一块数据。为节省内存，设置了在内存中最多存在 2 个下载完而又没有传给播放器的块。

一些细节：

  1. 该如何把数据传给播放器呢？我最初的设想是通过标准输出，这样简单好写。但 stdio 是无法 seek 的，这就意味着你只能从视频的开头看起，无法快进 :P 如你所见，现在的解决方案是用 HTTP 协议与播放器传输数据。需要快进的时候播放器发送 HTTP Range 请求，video_funnel 将请求中的范围经过分块、切片后“多线程”下载。但这样就又带来了两个问题：

    1. 需要播放器支持从 URL 播放。mplayer、mpv 之类的命令行播放器大多都支持，但一些 Windows 的播放器就不得而知了 :P 不过可以使用 HTML 的 video 标签在浏览器播放。
    2. 怎么就没有处理 Range 请求的包啊，自己处理很麻烦的好吗～

  2. 由于下载的部分是用异步 IO 写的，与播放器交互的服务器部分就不能使用 Flask 之类阻塞的框架了，幸好 aiohttp 居然同时支持客户端和服务端。

  3. 说起来简单，实际写起来处处是坑啊 :(

参加 https://www.v2ex.com/t/405569

GitHub: https://github.com/cshuaimin/video-funnel


Python中如何实现在线视频观看达到多线程下载的速度

29 回复

挺好的，要是能用 js 实现就更好了。


要实现多线程下载在线视频，核心思路是将视频文件分割成多个片段，用多个线程同时下载，最后合并。这里用requests和concurrent.futures演示一个基本方案。

import requests
import concurrent.futures
import os
from typing import List

def download_chunk(url: str, start: int, end: int, chunk_id: int) -> bytes:
    """下载指定范围的数据块"""
    headers = {'Range': f'bytes={start}-{end}'}
    response = requests.get(url, headers=headers, stream=True)
    response.raise_for_status()
    return chunk_id, response.content

def merge_chunks(chunks: List[bytes], output_path: str):
    """合并所有数据块"""
    with open(output_path, 'wb') as f:
        for chunk in sorted(chunks, key=lambda x: x[0]):
            f.write(chunk[1])

def threaded_video_download(url: str, output_path: str, num_threads: int = 4):
    """多线程下载视频主函数"""
    # 获取文件大小
    response = requests.head(url)
    file_size = int(response.headers.get('content-length', 0))
    
    if file_size == 0:
        raise ValueError("无法获取文件大小或服务器不支持Range请求")
    
    # 计算每个线程下载的字节范围
    chunk_size = file_size // num_threads
    ranges = []
    for i in range(num_threads):
        start = i * chunk_size
        end = start + chunk_size - 1 if i < num_threads - 1 else file_size - 1
        ranges.append((start, end, i))
    
    # 多线程下载
    chunks = []
    with concurrent.futures.ThreadPoolExecutor(max_workers=num_threads) as executor:
        futures = []
        for start, end, chunk_id in ranges:
            future = executor.submit(download_chunk, url, start, end, chunk_id)
            futures.append(future)
        
        for future in concurrent.futures.as_completed(futures):
            chunk_id, data = future.result()
            chunks.append((chunk_id, data))
            print(f"已下载数据块 {chunk_id + 1}/{num_threads}")
    
    # 合并文件
    merge_chunks(chunks, output_path)
    print(f"下载完成: {output_path}")

# 使用示例
if __name__ == "__main__":
    video_url = "https://example.com/video.mp4"  # 替换为实际视频URL
    output_file = "video.mp4"
    threaded_video_download(video_url, output_file, num_threads=8)

关键点说明：

  1. 用Range头请求特定字节范围，这是HTTP协议支持的功能
  2. 每个线程负责下载一个连续的数据块
  3. 用线程池管理并发下载
  4. 下载完成后按顺序合并数据块

注意：这个方案需要服务器支持Range请求（大多数视频服务器都支持）。实际使用时可能需要添加重试机制和进度显示。

总结：用Range请求+线程池实现分片并发下载。

可以可以，很实用啊感觉

$ pip install video_funnel
Collecting video_funnel
Could not find a version that satisfies the requirement video_funnel (from ver
sions: )
No matching distribution found for video_funnel

对啊，要是 js 实现的就可以直接在浏览器播放了

你是不是没有用官方源？可能是还没有同步过去

youtube 就是这个套路

应该是 pip 版本太老升级下

mo 一波 dalao 的 aiodl 真不错

大有用处。。。搞个视频网站

前排围观支持！！！

aria2c -x10 --stream-piece-selector=inorder

唉你看我这轮子造的，早知道有 aira2 这等神器还自己折腾什么。。。

不过 aira2 貌似不能快进？容我加上”边看边下“的功能 :)

mark.摩拜大佬

mark.摩拜大佬

收藏多，回复少，一群大佬准备大干

这个叫做 串流播放

膜拜大佬…

我也很奇怪为什么这么多收藏😂

mark.摩拜大佬

TS 流怎么搞……

m3u8 吗？那就没办法了……

Command “python setup.py egg_info” failed with error code 1 in C:\Users\UrAir\AppData\Local\Temp\pip-build-4koh515g\video-funnel<br>
怎么破

具体错误是什么呢？需要 Python 版本为 3.6，是不是这个原因？


> PS F:\myproject> pip install --user video_funnel
Collecting video_funnel
Using cached video_funnel-0.0.3.tar.gz
Complete output from command python setup.py egg_info:
Traceback (most recent call last):
File “<string>”, line 1, in <module>
File “C:\Users\UrAir\AppData\Local\Temp\pip-build-v519sou0\video-funnel<a target=”_blank" href=“http://setup.py” rel=“nofollow noopener”>setup.py", line 29, in <module>
‘vf = video_funnel.main:main’
File “c:\users\urair\appdata\local\programs\python\python36\lib\distutils<a target=”_blank" href=“http://core.py” rel=“nofollow noopener”>core.py", line 108, in setup
_setup_distribution = dist = klass(attrs)
File “c:\users\urair\appdata\local\programs\python\python36\lib\site-packages\setuptools<a target=”_blank" href=“http://dist.py” rel=“nofollow noopener”>dist.py", line 315, in init
self.fetch_build_eggs(attrs[‘setup_requires’])
File “c:\users\urair\appdata\local\programs\python\python36\lib\site-packages\setuptools<a target=”_blank" href=“http://dist.py” rel=“nofollow noopener”>dist.py", line 361, in fetch_build_eggs
replace_conflicting=True,
File “c:\users\urair\appdata\local\programs\python\python36\lib\site-packages\pkg_resources<a target=”_blank" href=“http://init.py” rel=“nofollow noopener”>init.py", line 850, in resolve
dist = best[req.key] = env.best_match(req, ws, installer)
File “c:\users\urair\appdata\local\programs\python\python36\lib\site-packages\pkg_resources<a target=”_blank" href=“http://init.py” rel=“nofollow noopener”>init.py", line 1122, in best_match
return self.obtain(req, installer)
File “c:\users\urair\appdata\local\programs\python\python36\lib\site-packages\pkg_resources<a target=”_blank" href=“http://init.py” rel=“nofollow noopener”>init.py", line 1134, in obtain
return installer(requirement)
File “c:\users\urair\appdata\local\programs\python\python36\lib\site-packages\setuptools<a target=”_blank" href=“http://dist.py” rel=“nofollow noopener”>dist.py", line 429, in fetch_build_egg
return cmd.easy_install(req)
File “c:\users\urair\appdata\local\programs\python\python36\lib\site-packages\setuptools\command<a target=”_blank" href=“http://easy_install.py” rel=“nofollow noopener”>easy_install.py", line 653, in easy_install
not self.always_copy, self.local_index
File “c:\users\urair\appdata\local\programs\python\python36\lib\site-packages\setuptools<a target=”_blank" href=“http://package_index.py” rel=“nofollow noopener”>package_index.py", line 636, in fetch_distribution
dist = find(requirement)
File “c:\users\urair\appdata\local\programs\python\python36\lib\site-packages\setuptools<a target=”_blank" href=“http://package_index.py” rel=“nofollow noopener”>package_index.py", line 617, in find
dist.download_location = self.download(dist.location, tmpdir)
File “c:\users\urair\appdata\local\programs\python\python36\lib\site-packages\setuptools<a target=”_blank" href=“http://package_index.py” rel=“nofollow noopener”>package_index.py", line 566, in download
found = self._download_url(scheme.group(1), spec, tmpdir)
File “c:\users\urair\appdata\local\programs\python\python36\lib\site-packages\setuptools<a target=”_blank" href=“http://package_index.py” rel=“nofollow noopener”>package_index.py", line 805, in _download_url
return self._attempt_download(url, filename)
File “c:\users\urair\appdata\local\programs\python\python36\lib\site-packages\setuptools<a target=”_blank" href=“http://package_index.py” rel=“nofollow noopener”>package_index.py", line 811, in _attempt_download
headers = self._download_to(url, filename)
File “c:\users\urair\appdata\local\programs\python\python36\lib\site-packages\setuptools<a target=”_blank" href=“http://package_index.py” rel=“nofollow noopener”>package_index.py", line 710, in _download_to
fp = self.open_url(strip_fragment(url))
File “c:\users\urair\appdata\local\programs\python\python36\lib\site-packages\setuptools<a target=”_blank" href=“http://package_index.py” rel=“nofollow noopener”>package_index.py", line 747, in open_url
return open_with_auth(url, self.opener)
File “c:\users\urair\appdata\local\programs\python\python36\lib\site-packages\setuptools<a target=”_blank" href=“http://package_index.py” rel=“nofollow noopener”>package_index.py", line 948, in _socket_timeout
return func(*args, **kwargs)
File “c:\users\urair\appdata\local\programs\python\python36\lib\site-packages\setuptools<a target=”_blank" href=“http://package_index.py” rel=“nofollow noopener”>package_index.py", line 1067, in open_with_auth
fp = opener(request)
File “c:\users\urair\appdata\local\programs\python\python36\lib\urllib<a target=”_blank" href=“http://request.py” rel=“nofollow noopener”>request.py", line 223, in urlopen
return opener.open(url, data, timeout)
File “c:\users\urair\appdata\local\programs\python\python36\lib\urllib<a target=”_blank" href=“http://request.py” rel=“nofollow noopener”>request.py", line 526, in open
response = self._open(req, data)
File “c:\users\urair\appdata\local\programs\python\python36\lib\urllib<a target=”_blank" href=“http://request.py” rel=“nofollow noopener”>request.py", line 544, in _open
‘_open’, req)
File “c:\users\urair\appdata\local\programs\python\python36\lib\urllib<a target=”_blank" href=“http://request.py” rel=“nofollow noopener”>request.py", line 504, in _call_chain
result = func(*args)
File “c:\users\urair\appdata\local\programs\python\python36\lib\urllib<a target=”_blank" href=“http://request.py” rel=“nofollow noopener”>request.py", line 1361, in https_open
context=self._context, check_hostname=self._check_hostname)
File “c:\users\urair\appdata\local\programs\python\python36\lib\urllib<a target=”_blank" href=“http://request.py” rel=“nofollow noopener”>request.py", line 1321, in do_open
r = h.getresponse()
File “c:\users\urair\appdata\local\programs\python\python36\lib\http<a target=”_blank" href=“http://client.py” rel=“nofollow noopener”>client.py", line 1331, in getresponse
response.begin()
File “c:\users\urair\appdata\local\programs\python\python36\lib\http<a target=”_blank" href=“http://client.py” rel=“nofollow noopener”>client.py", line 297, in begin
version, status, reason = self._read_status()
File “c:\users\urair\appdata\local\programs\python\python36\lib\http<a target=”_blank" href=“http://client.py” rel=“nofollow noopener”>client.py", line 258, in _read_status
line = str(self.fp.readline(_MAXLINE + 1), “iso-8859-1”)
File “c:\users\urair\appdata\local\programs\python\python36\lib<a target=”_blank" href=“http://socket.py” rel=“nofollow noopener”>socket.py", line 586, in readinto
return self._sock.recv_into(b)
File “c:\users\urair\appdata\local\programs\python\python36\lib<a target=”_blank" href=“http://ssl.py” rel=“nofollow noopener”>ssl.py", line 1009, in recv_into
return self.read(nbytes, buffer)
File “c:\users\urair\appdata\local\programs\python\python36\lib<a target=”_blank" href=“http://ssl.py” rel=“nofollow noopener”>ssl.py", line 871, in read
return self._sslobj.read(len, buffer)
File “c:\users\urair\appdata\local\programs\python\python36\lib<a target=”_blank" href=“http://ssl.py” rel=“nofollow noopener”>ssl.py", line 631, in read
v = self._sslobj.read(len, buffer)
socket.timeout: The read operation timed out

----------------------------------------
Command “python setup.py egg_info” failed with error code 1 in C:\Users\UrAir\AppData\Local\Temp\pip-build-v519sou0\video-funnel\

看最后一行，socket.timeout: The read operation timed out 应该是网络原因，换个 pip 源或挂代理试试

  • Listening at port 8080 …
    Error handling request
    Traceback (most recent call last):
    File “/root/.local/lib/python3.6/site-packages/aiohttp/web_protocol.py”, line 418, in start
    resp = await task
    File “/root/.local/lib/python3.6/site-packages/aiohttp/web_app.py”, line 458, in _handle
    resp = await handler(request)
    File “/root/.local/lib/python3.6/site-packages/video_funnel/init.py”, line 85, in handler
    del request.headers[‘Host’]
    TypeError: ‘multidict._multidict.CIMultiDictProxy’ object does not support item deletion
    貌似库改了

windows 下安装了找不到 vf 命令

–stream-piece-selector= inorder 和 geom 有什么区别？

回到顶部