镜像自地址
https://github.com/binary-husky/gpt_academic.git
已同步 2025-12-06 14:36:48 +00:00
比较提交
9 次代码提交
master-4.0
...
master
| 作者 | SHA1 | 提交日期 | |
|---|---|---|---|
|
|
0aa0472da4 | ||
|
|
3546f8c1c4 | ||
|
|
d8ad675f13 | ||
|
|
0ab0417954 | ||
|
|
248b0aefae | ||
|
|
661fe63941 | ||
|
|
269804fb82 | ||
|
|
8042750d41 | ||
|
|
65a4cf59c2 |
6
.dockerignore
普通文件
6
.dockerignore
普通文件
@@ -0,0 +1,6 @@
|
||||
.venv
|
||||
.github
|
||||
.vscode
|
||||
gpt_log
|
||||
tests
|
||||
README.md
|
||||
44
.github/workflows/build-with-chatglm.yml
vendored
44
.github/workflows/build-with-chatglm.yml
vendored
@@ -1,44 +0,0 @@
|
||||
# https://docs.github.com/en/actions/publishing-packages/publishing-docker-images#publishing-images-to-github-packages
|
||||
name: build-with-chatglm
|
||||
|
||||
on:
|
||||
push:
|
||||
branches:
|
||||
- 'master'
|
||||
|
||||
env:
|
||||
REGISTRY: ghcr.io
|
||||
IMAGE_NAME: ${{ github.repository }}_chatglm_moss
|
||||
|
||||
jobs:
|
||||
build-and-push-image:
|
||||
runs-on: ubuntu-latest
|
||||
permissions:
|
||||
contents: read
|
||||
packages: write
|
||||
|
||||
steps:
|
||||
- name: Checkout repository
|
||||
uses: actions/checkout@v3
|
||||
|
||||
- name: Log in to the Container registry
|
||||
uses: docker/login-action@v2
|
||||
with:
|
||||
registry: ${{ env.REGISTRY }}
|
||||
username: ${{ github.actor }}
|
||||
password: ${{ secrets.GITHUB_TOKEN }}
|
||||
|
||||
- name: Extract metadata (tags, labels) for Docker
|
||||
id: meta
|
||||
uses: docker/metadata-action@v4
|
||||
with:
|
||||
images: ${{ env.REGISTRY }}/${{ env.IMAGE_NAME }}
|
||||
|
||||
- name: Build and push Docker image
|
||||
uses: docker/build-push-action@v4
|
||||
with:
|
||||
context: .
|
||||
push: true
|
||||
file: docs/GithubAction+ChatGLM+Moss
|
||||
tags: ${{ steps.meta.outputs.tags }}
|
||||
labels: ${{ steps.meta.outputs.labels }}
|
||||
@@ -29,7 +29,7 @@ RUN python -c 'import loguru'
|
||||
|
||||
# 装载项目文件,安装剩余依赖(必要)
|
||||
COPY . .
|
||||
RUN uv venv --python=3.12 && uv pip install -r requirements.txt -i https://mirrors.aliyun.com/pypi/simple/
|
||||
RUN uv pip install -r requirements.txt -i https://mirrors.aliyun.com/pypi/simple/
|
||||
|
||||
# # 非必要步骤,用于预热模块(可以删除)
|
||||
RUN python -c 'from check_proxy import warm_up_modules; warm_up_modules()'
|
||||
|
||||
23
README.md
23
README.md
@@ -1,13 +1,12 @@
|
||||
> [!IMPORTANT]
|
||||
> `master主分支`最新动态(2025.3.2): 修复大量代码typo / 联网组件支持Jina的api / 增加deepseek-r1支持
|
||||
> `frontier开发分支`最新动态(2024.12.9): 更新对话时间线功能,优化xelatex论文翻译
|
||||
> `wiki文档`最新动态(2024.12.5): 更新ollama接入指南
|
||||
> `master主分支`最新动态(2025.8.23): Dockerfile构建效率大幅优化
|
||||
> `master主分支`最新动态(2025.7.31): 新GUI前端,Coming Soon
|
||||
>
|
||||
> 2025.2.2: 三分钟快速接入最强qwen2.5-max[视频](https://www.bilibili.com/video/BV1LeFuerEG4)
|
||||
> 2025.2.1: 支持自定义字体
|
||||
> 2024.10.10: 突发停电,紧急恢复了提供[whl包](https://drive.google.com/drive/folders/14kR-3V-lIbvGxri4AHc8TpiA1fqsw7SK?usp=sharing)的文件服务器
|
||||
> 2024.5.1: 加入Doc2x翻译PDF论文的功能,[查看详情](https://github.com/binary-husky/gpt_academic/wiki/Doc2x)
|
||||
> 2024.3.11: 全力支持Qwen、GLM、DeepseekCoder等中文大语言模型! SoVits语音克隆模块,[查看详情](https://www.bilibili.com/video/BV1Rp421S7tF/)
|
||||
> 2025.2.2: 三分钟快速接入最强qwen2.5-max[视频](https://www.bilibili.com/video/BV1LeFuerEG4)
|
||||
> 2025.2.1: 支持自定义字体
|
||||
> 2024.10.10: 突发停电,紧急恢复了提供[whl包](https://drive.google.com/drive/folders/14kR-3V-lIbvGxri4AHc8TpiA1fqsw7SK?usp=sharing)的文件服务器
|
||||
> 2024.5.1: 加入Doc2x翻译PDF论文的功能,[查看详情](https://github.com/binary-husky/gpt_academic/wiki/Doc2x)
|
||||
> 2024.3.11: 全力支持Qwen、GLM、DeepseekCoder等中文大语言模型! SoVits语音克隆模块,[查看详情](https://www.bilibili.com/video/BV1Rp421S7tF/)
|
||||
> 2024.1.17: 安装依赖时,请选择`requirements.txt`中**指定的版本**。 安装命令:`pip install -r requirements.txt`。
|
||||
|
||||
<br>
|
||||
@@ -64,7 +63,6 @@ Read this in [English](docs/README.English.md) | [日本語](docs/README.Japanes
|
||||
⭐支持mermaid图像渲染 | 支持让GPT生成[流程图](https://www.bilibili.com/video/BV18c41147H9/)、状态转移图、甘特图、饼状图、GitGraph等等(3.7版本)
|
||||
⭐Arxiv论文精细翻译 ([Docker](https://github.com/binary-husky/gpt_academic/pkgs/container/gpt_academic_with_latex)) | [插件] 一键[以超高质量翻译arxiv论文](https://www.bilibili.com/video/BV1dz4y1v77A/),目前最好的论文翻译工具
|
||||
⭐[实时语音对话输入](https://github.com/binary-husky/gpt_academic/blob/master/docs/use_audio.md) | [插件] 异步[监听音频](https://www.bilibili.com/video/BV1AV4y187Uy/),自动断句,自动寻找回答时机
|
||||
⭐AutoGen多智能体插件 | [插件] 借助微软AutoGen,探索多Agent的智能涌现可能!
|
||||
⭐虚空终端插件 | [插件] 能够使用自然语言直接调度本项目其他插件
|
||||
润色、翻译、代码解释 | 一键润色、翻译、查找论文语法错误、解释代码
|
||||
[自定义快捷键](https://www.bilibili.com/video/BV14s4y1E7jN) | 支持自定义快捷键
|
||||
@@ -172,6 +170,11 @@ flowchart TD
|
||||
conda create -n gptac_venv python=3.11 # 创建anaconda环境
|
||||
conda activate gptac_venv # 激活anaconda环境
|
||||
python -m pip install -r requirements.txt # 这个步骤和pip安装一样的步骤
|
||||
|
||||
# (选择III: 使用uv):
|
||||
uv venv --python=3.11 # 创建虚拟环境
|
||||
source ./.venv/bin/activate # 激活虚拟环境
|
||||
uv pip install --verbose -r requirements.txt -i https://mirrors.aliyun.com/pypi/simple/ # 安装依赖
|
||||
```
|
||||
|
||||
|
||||
@@ -359,7 +362,7 @@ Tip:不指定文件直接点击 `载入对话历史存档` 可以查看历史h
|
||||
|
||||
### II:版本:
|
||||
- version 3.80(TODO): 优化AutoGen插件主题并设计一系列衍生插件
|
||||
- version 3.70: 引入Mermaid绘图,实现GPT画脑图等功能
|
||||
- version 3.70: 引入Mermaid绘图,实现GPT画脑图等功能
|
||||
- version 3.60: 引入AutoGen作为新一代插件的基石
|
||||
- version 3.57: 支持GLM3,星火v3,文心一言v4,修复本地模型的并发BUG
|
||||
- version 3.56: 支持动态追加基础功能按钮,新汇报PDF汇总页面
|
||||
|
||||
@@ -3,8 +3,8 @@ from toolbox import trimmed_format_exc
|
||||
from loguru import logger
|
||||
|
||||
def get_crazy_functions():
|
||||
from crazy_functions.读文章写摘要 import 读文章写摘要
|
||||
from crazy_functions.生成函数注释 import 批量生成函数注释
|
||||
from crazy_functions.Paper_Abstract_Writer import Paper_Abstract_Writer
|
||||
from crazy_functions.Program_Comment_Gen import 批量Program_Comment_Gen
|
||||
from crazy_functions.SourceCode_Analyse import 解析项目本身
|
||||
from crazy_functions.SourceCode_Analyse import 解析一个Python项目
|
||||
from crazy_functions.SourceCode_Analyse import 解析一个Matlab项目
|
||||
@@ -17,26 +17,26 @@ def get_crazy_functions():
|
||||
from crazy_functions.高级功能函数模板 import 高阶功能模板函数
|
||||
from crazy_functions.高级功能函数模板 import Demo_Wrap
|
||||
from crazy_functions.Latex_Project_Polish import Latex英文润色
|
||||
from crazy_functions.询问多个大语言模型 import 同时问询
|
||||
from crazy_functions.Multi_LLM_Query import 同时问询
|
||||
from crazy_functions.SourceCode_Analyse import 解析一个Lua项目
|
||||
from crazy_functions.SourceCode_Analyse import 解析一个CSharp项目
|
||||
from crazy_functions.总结word文档 import 总结word文档
|
||||
from crazy_functions.解析JupyterNotebook import 解析ipynb文件
|
||||
from crazy_functions.Word_Summary import Word_Summary
|
||||
from crazy_functions.SourceCode_Analyse_JupyterNotebook import 解析ipynb文件
|
||||
from crazy_functions.Conversation_To_File import 载入对话历史存档
|
||||
from crazy_functions.Conversation_To_File import 对话历史存档
|
||||
from crazy_functions.Conversation_To_File import Conversation_To_File_Wrap
|
||||
from crazy_functions.Conversation_To_File import 删除所有本地对话历史记录
|
||||
from crazy_functions.辅助功能 import 清除缓存
|
||||
from crazy_functions.Helpers import 清除缓存
|
||||
from crazy_functions.Markdown_Translate import Markdown英译中
|
||||
from crazy_functions.批量总结PDF文档 import 批量总结PDF文档
|
||||
from crazy_functions.PDF_Summary import PDF_Summary
|
||||
from crazy_functions.PDF_Translate import 批量翻译PDF文档
|
||||
from crazy_functions.谷歌检索小助手 import 谷歌检索小助手
|
||||
from crazy_functions.理解PDF文档内容 import 理解PDF文档内容标准文件输入
|
||||
from crazy_functions.Google_Scholar_Assistant_Legacy import Google_Scholar_Assistant_Legacy
|
||||
from crazy_functions.PDF_QA import PDF_QA标准文件输入
|
||||
from crazy_functions.Latex_Project_Polish import Latex中文润色
|
||||
from crazy_functions.Latex_Project_Polish import Latex英文纠错
|
||||
from crazy_functions.Markdown_Translate import Markdown中译英
|
||||
from crazy_functions.虚空终端 import 虚空终端
|
||||
from crazy_functions.生成多种Mermaid图表 import Mermaid_Gen
|
||||
from crazy_functions.Void_Terminal import Void_Terminal
|
||||
from crazy_functions.Mermaid_Figure_Gen import Mermaid_Gen
|
||||
from crazy_functions.PDF_Translate_Wrap import PDF_Tran
|
||||
from crazy_functions.Latex_Function import Latex英文纠错加PDF对比
|
||||
from crazy_functions.Latex_Function import Latex翻译中文并重新编译PDF
|
||||
@@ -67,7 +67,7 @@ def get_crazy_functions():
|
||||
"Color": "stop",
|
||||
"AsButton": True,
|
||||
"Info": "使用自然语言实现您的想法",
|
||||
"Function": HotReload(虚空终端),
|
||||
"Function": HotReload(Void_Terminal),
|
||||
},
|
||||
"解析整个Python项目": {
|
||||
"Group": "编程",
|
||||
@@ -125,7 +125,7 @@ def get_crazy_functions():
|
||||
"Color": "stop",
|
||||
"AsButton": False,
|
||||
"Info": "批量总结word文档 | 输入参数为路径",
|
||||
"Function": HotReload(总结word文档),
|
||||
"Function": HotReload(Word_Summary),
|
||||
},
|
||||
"解析整个Matlab项目": {
|
||||
"Group": "编程",
|
||||
@@ -204,7 +204,7 @@ def get_crazy_functions():
|
||||
"Color": "stop",
|
||||
"AsButton": False,
|
||||
"Info": "读取Tex论文并写摘要 | 输入参数为路径",
|
||||
"Function": HotReload(读文章写摘要),
|
||||
"Function": HotReload(Paper_Abstract_Writer),
|
||||
},
|
||||
"翻译README或MD": {
|
||||
"Group": "编程",
|
||||
@@ -225,14 +225,14 @@ def get_crazy_functions():
|
||||
"Color": "stop",
|
||||
"AsButton": False, # 加入下拉菜单中
|
||||
"Info": "批量生成函数的注释 | 输入参数为路径",
|
||||
"Function": HotReload(批量生成函数注释),
|
||||
"Function": HotReload(批量Program_Comment_Gen),
|
||||
},
|
||||
"保存当前的对话": {
|
||||
"Group": "对话",
|
||||
"Color": "stop",
|
||||
"AsButton": True,
|
||||
"Info": "保存当前的对话 | 不需要输入参数",
|
||||
"Function": HotReload(对话历史存档), # 当注册Class后,Function旧接口仅会在“虚空终端”中起作用
|
||||
"Function": HotReload(对话历史存档), # 当注册Class后,Function旧接口仅会在“Void_Terminal”中起作用
|
||||
"Class": Conversation_To_File_Wrap # 新一代插件需要注册Class
|
||||
},
|
||||
"[多线程Demo]解析此项目本身(源码自译解)": {
|
||||
@@ -258,12 +258,12 @@ def get_crazy_functions():
|
||||
"Function": None,
|
||||
"Class": Demo_Wrap, # 新一代插件需要注册Class
|
||||
},
|
||||
"精准翻译PDF论文": {
|
||||
"PDF论文翻译": {
|
||||
"Group": "学术",
|
||||
"Color": "stop",
|
||||
"AsButton": True,
|
||||
"Info": "精准翻译PDF论文为中文 | 输入参数为路径",
|
||||
"Function": HotReload(批量翻译PDF文档), # 当注册Class后,Function旧接口仅会在“虚空终端”中起作用
|
||||
"Function": HotReload(批量翻译PDF文档), # 当注册Class后,Function旧接口仅会在“Void_Terminal”中起作用
|
||||
"Class": PDF_Tran, # 新一代插件需要注册Class
|
||||
},
|
||||
"询问多个GPT模型": {
|
||||
@@ -277,21 +277,21 @@ def get_crazy_functions():
|
||||
"Color": "stop",
|
||||
"AsButton": False, # 加入下拉菜单中
|
||||
"Info": "批量总结PDF文档的内容 | 输入参数为路径",
|
||||
"Function": HotReload(批量总结PDF文档),
|
||||
"Function": HotReload(PDF_Summary),
|
||||
},
|
||||
"谷歌学术检索助手(输入谷歌学术搜索页url)": {
|
||||
"Group": "学术",
|
||||
"Color": "stop",
|
||||
"AsButton": False, # 加入下拉菜单中
|
||||
"Info": "使用谷歌学术检索助手搜索指定URL的结果 | 输入参数为谷歌学术搜索页的URL",
|
||||
"Function": HotReload(谷歌检索小助手),
|
||||
"Function": HotReload(Google_Scholar_Assistant_Legacy),
|
||||
},
|
||||
"理解PDF文档内容 (模仿ChatPDF)": {
|
||||
"Group": "学术",
|
||||
"Color": "stop",
|
||||
"AsButton": False, # 加入下拉菜单中
|
||||
"Info": "理解PDF文档的内容并进行回答 | 输入参数为路径",
|
||||
"Function": HotReload(理解PDF文档内容标准文件输入),
|
||||
"Function": HotReload(PDF_QA标准文件输入),
|
||||
},
|
||||
"英文Latex项目全文润色(输入路径或上传压缩包)": {
|
||||
"Group": "学术",
|
||||
@@ -356,7 +356,7 @@ def get_crazy_functions():
|
||||
r"例如当单词'agent'翻译不准确时, 请尝试把以下指令复制到高级参数区: "
|
||||
r'If the term "agent" is used in this section, it should be translated to "智能体". ',
|
||||
"Info": "ArXiv论文精细翻译 | 输入参数arxiv论文的ID,比如1812.10695",
|
||||
"Function": HotReload(Latex翻译中文并重新编译PDF), # 当注册Class后,Function旧接口仅会在“虚空终端”中起作用
|
||||
"Function": HotReload(Latex翻译中文并重新编译PDF), # 当注册Class后,Function旧接口仅会在“Void_Terminal”中起作用
|
||||
"Class": Arxiv_Localize, # 新一代插件需要注册Class
|
||||
},
|
||||
"📚本地Latex论文精细翻译(上传Latex项目)[需Latex]": {
|
||||
@@ -379,7 +379,7 @@ def get_crazy_functions():
|
||||
r"例如当单词'agent'翻译不准确时, 请尝试把以下指令复制到高级参数区: "
|
||||
r'If the term "agent" is used in this section, it should be translated to "智能体". ',
|
||||
"Info": "PDF翻译中文,并重新编译PDF | 输入参数为路径",
|
||||
"Function": HotReload(PDF翻译中文并重新编译PDF), # 当注册Class后,Function旧接口仅会在“虚空终端”中起作用
|
||||
"Function": HotReload(PDF翻译中文并重新编译PDF), # 当注册Class后,Function旧接口仅会在“Void_Terminal”中起作用
|
||||
"Class": PDF_Localize # 新一代插件需要注册Class
|
||||
},
|
||||
"批量文件询问 (支持自定义总结各种文件)": {
|
||||
@@ -400,7 +400,7 @@ def get_crazy_functions():
|
||||
"Color": "stop",
|
||||
"AsButton": False,
|
||||
"Info": "使用 DALLE2/DALLE3 生成图片 | 输入参数字符串,提供图像的内容",
|
||||
"Function": HotReload(图片生成_DALLE2), # 当注册Class后,Function旧接口仅会在“虚空终端”中起作用
|
||||
"Function": HotReload(图片生成_DALLE2), # 当注册Class后,Function旧接口仅会在“Void_Terminal”中起作用
|
||||
"Class": ImageGen_Wrap # 新一代插件需要注册Class
|
||||
},
|
||||
}
|
||||
@@ -427,7 +427,7 @@ def get_crazy_functions():
|
||||
|
||||
|
||||
try:
|
||||
from crazy_functions.下载arxiv论文翻译摘要 import 下载arxiv论文并翻译摘要
|
||||
from crazy_functions.Arxiv_Downloader import 下载arxiv论文并翻译摘要
|
||||
|
||||
function_plugins.update(
|
||||
{
|
||||
@@ -465,7 +465,7 @@ def get_crazy_functions():
|
||||
logger.error("Load function plugin failed")
|
||||
|
||||
try:
|
||||
from crazy_functions.询问多个大语言模型 import 同时问询_指定模型
|
||||
from crazy_functions.Multi_LLM_Query import 同时问询_指定模型
|
||||
|
||||
function_plugins.update(
|
||||
{
|
||||
@@ -486,7 +486,7 @@ def get_crazy_functions():
|
||||
|
||||
|
||||
try:
|
||||
from crazy_functions.总结音视频 import 总结音视频
|
||||
from crazy_functions.Audio_Summary import Audio_Summary
|
||||
|
||||
function_plugins.update(
|
||||
{
|
||||
@@ -497,7 +497,7 @@ def get_crazy_functions():
|
||||
"AdvancedArgs": True,
|
||||
"ArgsReminder": "调用openai api 使用whisper-1模型, 目前支持的格式:mp4, m4a, wav, mpga, mpeg, mp3。此处可以输入解析提示,例如:解析为简体中文(默认)。",
|
||||
"Info": "批量总结音频或视频 | 输入参数为路径",
|
||||
"Function": HotReload(总结音视频),
|
||||
"Function": HotReload(Audio_Summary),
|
||||
}
|
||||
}
|
||||
)
|
||||
@@ -506,7 +506,7 @@ def get_crazy_functions():
|
||||
logger.error("Load function plugin failed")
|
||||
|
||||
try:
|
||||
from crazy_functions.数学动画生成manim import 动画生成
|
||||
from crazy_functions.Math_Animation_Gen import 动画生成
|
||||
|
||||
function_plugins.update(
|
||||
{
|
||||
@@ -543,7 +543,7 @@ def get_crazy_functions():
|
||||
logger.error("Load function plugin failed")
|
||||
|
||||
try:
|
||||
from crazy_functions.知识库问答 import 知识库文件注入
|
||||
from crazy_functions.Vectorstore_QA import 知识库文件注入
|
||||
|
||||
function_plugins.update(
|
||||
{
|
||||
@@ -562,7 +562,7 @@ def get_crazy_functions():
|
||||
logger.error("Load function plugin failed")
|
||||
|
||||
try:
|
||||
from crazy_functions.知识库问答 import 读取知识库作答
|
||||
from crazy_functions.Vectorstore_QA import 读取知识库作答
|
||||
|
||||
function_plugins.update(
|
||||
{
|
||||
@@ -581,7 +581,7 @@ def get_crazy_functions():
|
||||
logger.error("Load function plugin failed")
|
||||
|
||||
try:
|
||||
from crazy_functions.交互功能函数模板 import 交互功能模板函数
|
||||
from crazy_functions.Interactive_Func_Template import 交互功能模板函数
|
||||
|
||||
function_plugins.update(
|
||||
{
|
||||
@@ -603,7 +603,7 @@ def get_crazy_functions():
|
||||
|
||||
ENABLE_AUDIO = get_conf("ENABLE_AUDIO")
|
||||
if ENABLE_AUDIO:
|
||||
from crazy_functions.语音助手 import 语音助手
|
||||
from crazy_functions.Audio_Assistant import Audio_Assistant
|
||||
|
||||
function_plugins.update(
|
||||
{
|
||||
@@ -612,7 +612,7 @@ def get_crazy_functions():
|
||||
"Color": "stop",
|
||||
"AsButton": True,
|
||||
"Info": "这是一个时刻聆听着的语音对话助手 | 没有输入参数",
|
||||
"Function": HotReload(语音助手),
|
||||
"Function": HotReload(Audio_Assistant),
|
||||
}
|
||||
}
|
||||
)
|
||||
@@ -621,7 +621,7 @@ def get_crazy_functions():
|
||||
logger.error("Load function plugin failed")
|
||||
|
||||
try:
|
||||
from crazy_functions.批量翻译PDF文档_NOUGAT import 批量翻译PDF文档
|
||||
from crazy_functions.PDF_Translate_Nougat import 批量翻译PDF文档
|
||||
|
||||
function_plugins.update(
|
||||
{
|
||||
@@ -638,7 +638,7 @@ def get_crazy_functions():
|
||||
logger.error("Load function plugin failed")
|
||||
|
||||
try:
|
||||
from crazy_functions.函数动态生成 import 函数动态生成
|
||||
from crazy_functions.Dynamic_Function_Generate import Dynamic_Function_Generate
|
||||
|
||||
function_plugins.update(
|
||||
{
|
||||
@@ -646,7 +646,7 @@ def get_crazy_functions():
|
||||
"Group": "智能体",
|
||||
"Color": "stop",
|
||||
"AsButton": False,
|
||||
"Function": HotReload(函数动态生成),
|
||||
"Function": HotReload(Dynamic_Function_Generate),
|
||||
}
|
||||
}
|
||||
)
|
||||
@@ -655,14 +655,14 @@ def get_crazy_functions():
|
||||
logger.error("Load function plugin failed")
|
||||
|
||||
# try:
|
||||
# from crazy_functions.多智能体 import 多智能体终端
|
||||
# from crazy_functions.Multi_Agent_Legacy import Multi_Agent_Legacy终端
|
||||
# function_plugins.update(
|
||||
# {
|
||||
# "AutoGen多智能体终端(仅供测试)": {
|
||||
# "AutoGenMulti_Agent_Legacy终端(仅供测试)": {
|
||||
# "Group": "智能体",
|
||||
# "Color": "stop",
|
||||
# "AsButton": False,
|
||||
# "Function": HotReload(多智能体终端),
|
||||
# "Function": HotReload(Multi_Agent_Legacy终端),
|
||||
# }
|
||||
# }
|
||||
# )
|
||||
@@ -670,23 +670,6 @@ def get_crazy_functions():
|
||||
# logger.error(trimmed_format_exc())
|
||||
# logger.error("Load function plugin failed")
|
||||
|
||||
try:
|
||||
from crazy_functions.互动小游戏 import 随机小游戏
|
||||
|
||||
function_plugins.update(
|
||||
{
|
||||
"随机互动小游戏(仅供测试)": {
|
||||
"Group": "智能体",
|
||||
"Color": "stop",
|
||||
"AsButton": False,
|
||||
"Function": HotReload(随机小游戏),
|
||||
}
|
||||
}
|
||||
)
|
||||
except:
|
||||
logger.error(trimmed_format_exc())
|
||||
logger.error("Load function plugin failed")
|
||||
|
||||
try:
|
||||
from crazy_functions.Rag_Interface import Rag问答
|
||||
|
||||
@@ -726,22 +709,22 @@ def get_crazy_functions():
|
||||
|
||||
|
||||
|
||||
# try:
|
||||
# from crazy_functions.Paper_Reading import 快速论文解读
|
||||
# function_plugins.update(
|
||||
# {
|
||||
# "速读论文": {
|
||||
# "Group": "学术",
|
||||
# "Color": "stop",
|
||||
# "AsButton": False,
|
||||
# "Info": "上传一篇论文进行快速分析和解读 | 输入参数为论文路径或DOI/arXiv ID",
|
||||
# "Function": HotReload(快速论文解读),
|
||||
# },
|
||||
# }
|
||||
# )
|
||||
# except:
|
||||
# logger.error(trimmed_format_exc())
|
||||
# logger.error("Load function plugin failed")
|
||||
try:
|
||||
from crazy_functions.Paper_Reading import 快速论文解读
|
||||
function_plugins.update(
|
||||
{
|
||||
"速读论文": {
|
||||
"Group": "学术",
|
||||
"Color": "stop",
|
||||
"AsButton": False,
|
||||
"Info": "上传一篇论文进行快速分析和解读 | 输入参数为论文路径或DOI/arXiv ID",
|
||||
"Function": HotReload(快速论文解读),
|
||||
},
|
||||
}
|
||||
)
|
||||
except:
|
||||
logger.error(trimmed_format_exc())
|
||||
logger.error("Load function plugin failed")
|
||||
|
||||
|
||||
# try:
|
||||
|
||||
@@ -168,7 +168,7 @@ class InterviewAssistant(AliyunASR):
|
||||
|
||||
|
||||
@CatchException
|
||||
def 语音助手(txt, llm_kwargs, plugin_kwargs, chatbot, history, system_prompt, user_request):
|
||||
def Audio_Assistant(txt, llm_kwargs, plugin_kwargs, chatbot, history, system_prompt, user_request):
|
||||
# pip install -U openai-whisper
|
||||
chatbot.append(["对话助手函数插件:使用时,双手离开鼠标键盘吧", "音频助手, 正在听您讲话(点击“停止”键可终止程序)..."])
|
||||
yield from update_ui(chatbot=chatbot, history=history) # 刷新界面
|
||||
@@ -132,13 +132,13 @@ def AnalyAudio(parse_prompt, file_manifest, llm_kwargs, chatbot, history):
|
||||
|
||||
|
||||
@CatchException
|
||||
def 总结音视频(txt, llm_kwargs, plugin_kwargs, chatbot, history, system_prompt, WEB_PORT):
|
||||
def Audio_Summary(txt, llm_kwargs, plugin_kwargs, chatbot, history, system_prompt, WEB_PORT):
|
||||
import glob, os
|
||||
|
||||
# 基本信息:功能、贡献者
|
||||
chatbot.append([
|
||||
"函数插件功能?",
|
||||
"总结音视频内容,函数插件贡献者: dalvqw & BinaryHusky"])
|
||||
"Audio_Summary内容,函数插件贡献者: dalvqw & BinaryHusky"])
|
||||
yield from update_ui(chatbot=chatbot, history=history) # 刷新界面
|
||||
|
||||
try:
|
||||
@@ -4,7 +4,7 @@ from crazy_functions.crazy_utils import input_clipping
|
||||
import copy, json
|
||||
|
||||
@CatchException
|
||||
def 命令行助手(txt, llm_kwargs, plugin_kwargs, chatbot, history, system_prompt, user_request):
|
||||
def Commandline_Assistant(txt, llm_kwargs, plugin_kwargs, chatbot, history, system_prompt, user_request):
|
||||
"""
|
||||
txt 输入栏用户输入的文本, 例如需要翻译的一段话, 再例如一个包含了待处理文件的路径
|
||||
llm_kwargs gpt模型参数, 如温度和top_p等, 一般原样传递下去就行
|
||||
@@ -139,7 +139,7 @@ def get_recent_file_prompt_support(chatbot):
|
||||
return path
|
||||
|
||||
@CatchException
|
||||
def 函数动态生成(txt, llm_kwargs, plugin_kwargs, chatbot, history, system_prompt, user_request):
|
||||
def Dynamic_Function_Generate(txt, llm_kwargs, plugin_kwargs, chatbot, history, system_prompt, user_request):
|
||||
"""
|
||||
txt 输入栏用户输入的文本,例如需要翻译的一段话,再例如一个包含了待处理文件的路径
|
||||
llm_kwargs gpt模型参数,如温度和top_p等,一般原样传递下去就行
|
||||
@@ -159,7 +159,7 @@ def 函数动态生成(txt, llm_kwargs, plugin_kwargs, chatbot, history, system_
|
||||
|
||||
# ⭐ 文件上传区是否有东西
|
||||
# 1. 如果有文件: 作为函数参数
|
||||
# 2. 如果没有文件:需要用GPT提取参数 (太懒了,以后再写,虚空终端已经实现了类似的代码)
|
||||
# 2. 如果没有文件:需要用GPT提取参数 (太懒了,以后再写,Void_Terminal已经实现了类似的代码)
|
||||
file_list = []
|
||||
if get_plugin_arg(plugin_kwargs, key="file_path_arg", default=False):
|
||||
file_path = get_plugin_arg(plugin_kwargs, key="file_path_arg", default=None)
|
||||
@@ -132,7 +132,7 @@ def get_meta_information(url, chatbot, history):
|
||||
return profile
|
||||
|
||||
@CatchException
|
||||
def 谷歌检索小助手(txt, llm_kwargs, plugin_kwargs, chatbot, history, system_prompt, user_request):
|
||||
def Google_Scholar_Assistant_Legacy(txt, llm_kwargs, plugin_kwargs, chatbot, history, system_prompt, user_request):
|
||||
disable_auto_promotion(chatbot=chatbot)
|
||||
# 基本信息:功能、贡献者
|
||||
chatbot.append([
|
||||
@@ -13,13 +13,13 @@ def 交互功能模板函数(txt, llm_kwargs, plugin_kwargs, chatbot, history, s
|
||||
user_request 当前用户的请求信息(IP地址等)
|
||||
"""
|
||||
history = [] # 清空历史,以免输入溢出
|
||||
chatbot.append(("这是什么功能?", "交互功能函数模板。在执行完成之后, 可以将自身的状态存储到cookie中, 等待用户的再次调用。"))
|
||||
chatbot.append(("这是什么功能?", "Interactive_Func_Template。在执行完成之后, 可以将自身的状态存储到cookie中, 等待用户的再次调用。"))
|
||||
yield from update_ui(chatbot=chatbot, history=history) # 刷新界面
|
||||
|
||||
state = chatbot._cookies.get('plugin_state_0001', None) # 初始化插件状态
|
||||
|
||||
if state is None:
|
||||
chatbot._cookies['lock_plugin'] = 'crazy_functions.交互功能函数模板->交互功能模板函数' # 赋予插件锁定 锁定插件回调路径,当下一次用户提交时,会直接转到该函数
|
||||
chatbot._cookies['lock_plugin'] = 'crazy_functions.Interactive_Func_Template->交互功能模板函数' # 赋予插件锁定 锁定插件回调路径,当下一次用户提交时,会直接转到该函数
|
||||
chatbot._cookies['plugin_state_0001'] = 'wait_user_keyword' # 赋予插件状态
|
||||
|
||||
chatbot.append(("第一次调用:", "请输入关键词, 我将为您查找相关壁纸, 建议使用英文单词, 插件锁定中,请直接提交即可。"))
|
||||
@@ -16,7 +16,7 @@ def 随机小游戏(prompt, llm_kwargs, plugin_kwargs, chatbot, history, system_
|
||||
llm_kwargs,
|
||||
cls,
|
||||
plugin_name='MiniGame_ResumeStory',
|
||||
callback_fn='crazy_functions.互动小游戏->随机小游戏',
|
||||
callback_fn='crazy_functions.Interactive_Mini_Game->随机小游戏',
|
||||
lock_plugin=True
|
||||
)
|
||||
yield from state.continue_game(prompt, chatbot, history)
|
||||
@@ -34,7 +34,7 @@ def 随机小游戏1(prompt, llm_kwargs, plugin_kwargs, chatbot, history, system
|
||||
llm_kwargs,
|
||||
cls,
|
||||
plugin_name='MiniGame_ASCII_Art',
|
||||
callback_fn='crazy_functions.互动小游戏->随机小游戏1',
|
||||
callback_fn='crazy_functions.Interactive_Mini_Game->随机小游戏1',
|
||||
lock_plugin=True
|
||||
)
|
||||
yield from state.continue_game(prompt, chatbot, history)
|
||||
@@ -297,7 +297,7 @@ def 解析历史输入(history, llm_kwargs, file_manifest, chatbot, plugin_kwarg
|
||||
|
||||
|
||||
@CatchException
|
||||
def 生成多种Mermaid图表(
|
||||
def Mermaid_Figure_Gen(
|
||||
txt, llm_kwargs, plugin_kwargs, chatbot, history, system_prompt, web_port
|
||||
):
|
||||
"""
|
||||
@@ -426,7 +426,7 @@ class Mermaid_Gen(GptAcademicPluginTemplate):
|
||||
"思维导图",
|
||||
]
|
||||
plugin_kwargs = options.index(plugin_kwargs['Type_of_Mermaid'])
|
||||
yield from 生成多种Mermaid图表(
|
||||
yield from Mermaid_Figure_Gen(
|
||||
txt,
|
||||
llm_kwargs,
|
||||
plugin_kwargs,
|
||||
@@ -22,7 +22,7 @@ def remove_model_prefix(llm):
|
||||
|
||||
|
||||
@CatchException
|
||||
def 多智能体终端(txt, llm_kwargs, plugin_kwargs, chatbot, history, system_prompt, user_request):
|
||||
def Multi_Agent_Legacy终端(txt, llm_kwargs, plugin_kwargs, chatbot, history, system_prompt, user_request):
|
||||
"""
|
||||
txt 输入栏用户输入的文本,例如需要翻译的一段话,再例如一个包含了待处理文件的路径
|
||||
llm_kwargs gpt模型参数,如温度和top_p等,一般原样传递下去就行
|
||||
@@ -78,17 +78,17 @@ def 多智能体终端(txt, llm_kwargs, plugin_kwargs, chatbot, history, system_
|
||||
chatbot.get_cookies()['lock_plugin'] = None
|
||||
persistent_class_multi_user_manager = GradioMultiuserManagerForPersistentClasses()
|
||||
user_uuid = chatbot.get_cookies().get('uuid')
|
||||
persistent_key = f"{user_uuid}->多智能体终端"
|
||||
persistent_key = f"{user_uuid}->Multi_Agent_Legacy终端"
|
||||
if persistent_class_multi_user_manager.already_alive(persistent_key):
|
||||
# 当已经存在一个正在运行的多智能体终端时,直接将用户输入传递给它,而不是再次启动一个新的多智能体终端
|
||||
# 当已经存在一个正在运行的Multi_Agent_Legacy终端时,直接将用户输入传递给它,而不是再次启动一个新的Multi_Agent_Legacy终端
|
||||
logger.info('[debug] feed new user input')
|
||||
executor = persistent_class_multi_user_manager.get(persistent_key)
|
||||
exit_reason = yield from executor.main_process_ui_control(txt, create_or_resume="resume")
|
||||
else:
|
||||
# 运行多智能体终端 (首次)
|
||||
# 运行Multi_Agent_Legacy终端 (首次)
|
||||
logger.info('[debug] create new executor instance')
|
||||
history = []
|
||||
chatbot.append(["正在启动: 多智能体终端", "插件动态生成, 执行开始, 作者 Microsoft & Binary-Husky."])
|
||||
chatbot.append(["正在启动: Multi_Agent_Legacy终端", "插件动态生成, 执行开始, 作者 Microsoft & Binary-Husky."])
|
||||
yield from update_ui(chatbot=chatbot, history=history) # 刷新界面
|
||||
executor = AutoGenMath(llm_kwargs, plugin_kwargs, chatbot, history, system_prompt, user_request)
|
||||
persistent_class_multi_user_manager.set(persistent_key, executor)
|
||||
@@ -96,7 +96,7 @@ def 多智能体终端(txt, llm_kwargs, plugin_kwargs, chatbot, history, system_
|
||||
|
||||
if exit_reason == "wait_feedback":
|
||||
# 当用户点击了“等待反馈”按钮时,将executor存储到cookie中,等待用户的再次调用
|
||||
executor.chatbot.get_cookies()['lock_plugin'] = 'crazy_functions.多智能体->多智能体终端'
|
||||
executor.chatbot.get_cookies()['lock_plugin'] = 'crazy_functions.Multi_Agent_Legacy->Multi_Agent_Legacy终端'
|
||||
else:
|
||||
executor.chatbot.get_cookies()['lock_plugin'] = None
|
||||
yield from update_ui(chatbot=executor.chatbot, history=executor.history) # 更新状态
|
||||
@@ -62,7 +62,7 @@ def 解析PDF(file_name, llm_kwargs, plugin_kwargs, chatbot, history, system_pro
|
||||
|
||||
|
||||
@CatchException
|
||||
def 理解PDF文档内容标准文件输入(txt, llm_kwargs, plugin_kwargs, chatbot, history, system_prompt, user_request):
|
||||
def PDF_QA标准文件输入(txt, llm_kwargs, plugin_kwargs, chatbot, history, system_prompt, user_request):
|
||||
import glob, os
|
||||
|
||||
# 基本信息:功能、贡献者
|
||||
@@ -103,13 +103,13 @@ do not have too much repetitive information, numerical values using the original
|
||||
|
||||
|
||||
@CatchException
|
||||
def 批量总结PDF文档(txt, llm_kwargs, plugin_kwargs, chatbot, history, system_prompt, user_request):
|
||||
def PDF_Summary(txt, llm_kwargs, plugin_kwargs, chatbot, history, system_prompt, user_request):
|
||||
import glob, os
|
||||
|
||||
# 基本信息:功能、贡献者
|
||||
chatbot.append([
|
||||
"函数插件功能?",
|
||||
"批量总结PDF文档。函数插件贡献者: ValeriaWong,Eralien"])
|
||||
"PDF_Summary。函数插件贡献者: ValeriaWong,Eralien"])
|
||||
yield from update_ui(chatbot=chatbot, history=history) # 刷新界面
|
||||
|
||||
# 尝试导入依赖,如果缺少依赖,则给出安装建议
|
||||
@@ -43,7 +43,7 @@ def 解析Paper(file_manifest, project_folder, llm_kwargs, plugin_kwargs, chatbo
|
||||
|
||||
|
||||
@CatchException
|
||||
def 读文章写摘要(txt, llm_kwargs, plugin_kwargs, chatbot, history, system_prompt, user_request):
|
||||
def Paper_Abstract_Writer(txt, llm_kwargs, plugin_kwargs, chatbot, history, system_prompt, user_request):
|
||||
history = [] # 清空历史,以免输入溢出
|
||||
import glob, os
|
||||
if os.path.exists(txt):
|
||||
@@ -4,7 +4,7 @@ from toolbox import CatchException, report_exception
|
||||
from toolbox import write_history_to_file, promote_file_to_downloadzone
|
||||
from crazy_functions.crazy_utils import request_gpt_model_in_new_thread_with_ui_alive
|
||||
|
||||
def 生成函数注释(file_manifest, project_folder, llm_kwargs, plugin_kwargs, chatbot, history, system_prompt):
|
||||
def Program_Comment_Gen(file_manifest, project_folder, llm_kwargs, plugin_kwargs, chatbot, history, system_prompt):
|
||||
import time, os
|
||||
logger.info('begin analysis on:', file_manifest)
|
||||
for index, fp in enumerate(file_manifest):
|
||||
@@ -34,7 +34,7 @@ def 生成函数注释(file_manifest, project_folder, llm_kwargs, plugin_kwargs,
|
||||
|
||||
|
||||
@CatchException
|
||||
def 批量生成函数注释(txt, llm_kwargs, plugin_kwargs, chatbot, history, system_prompt, user_request):
|
||||
def 批量Program_Comment_Gen(txt, llm_kwargs, plugin_kwargs, chatbot, history, system_prompt, user_request):
|
||||
history = [] # 清空历史,以免输入溢出
|
||||
import glob, os
|
||||
if os.path.exists(txt):
|
||||
@@ -51,4 +51,4 @@ def 批量生成函数注释(txt, llm_kwargs, plugin_kwargs, chatbot, history, s
|
||||
report_exception(chatbot, history, a = f"解析项目: {txt}", b = f"找不到任何.tex文件: {txt}")
|
||||
yield from update_ui(chatbot=chatbot, history=history) # 刷新界面
|
||||
return
|
||||
yield from 生成函数注释(file_manifest, project_folder, llm_kwargs, plugin_kwargs, chatbot, history, system_prompt)
|
||||
yield from Program_Comment_Gen(file_manifest, project_folder, llm_kwargs, plugin_kwargs, chatbot, history, system_prompt)
|
||||
@@ -79,8 +79,8 @@ def 知识库文件注入(txt, llm_kwargs, plugin_kwargs, chatbot, history, syst
|
||||
# yield from update_ui(chatbot=chatbot, history=history) # 刷新界面
|
||||
# chatbot._cookies['langchain_plugin_embedding'] = kai.get_current_archive_id()
|
||||
# chatbot._cookies['lock_plugin'] = 'crazy_functions.知识库文件注入->读取知识库作答'
|
||||
# chatbot.append(['完成', "“根据知识库作答”函数插件已经接管问答系统, 提问吧! 但注意, 您接下来不能再使用其他插件了,刷新页面即可以退出知识库问答模式。"])
|
||||
chatbot.append(['构建完成', f"当前知识库内的有效文件:\n\n---\n\n{kai_files}\n\n---\n\n请切换至“知识库问答”插件进行知识库访问, 或者使用此插件继续上传更多文件。"])
|
||||
# chatbot.append(['完成', "“根据知识库作答”函数插件已经接管问答系统, 提问吧! 但注意, 您接下来不能再使用其他插件了,刷新页面即可以退出Vectorstore_QA模式。"])
|
||||
chatbot.append(['构建完成', f"当前知识库内的有效文件:\n\n---\n\n{kai_files}\n\n---\n\n请切换至“Vectorstore_QA”插件进行知识库访问, 或者使用此插件继续上传更多文件。"])
|
||||
yield from update_ui(chatbot=chatbot, history=history) # 刷新界面 # 由于请求gpt需要一段时间,我们先及时地做一次界面更新
|
||||
|
||||
@CatchException
|
||||
@@ -21,7 +21,7 @@ Please describe in natural language what you want to do.
|
||||
5. If you don't need to upload a file, you can simply repeat your command again.
|
||||
"""
|
||||
explain_msg = """
|
||||
## 虚空终端插件说明:
|
||||
## Void_Terminal插件说明:
|
||||
|
||||
1. 请用**自然语言**描述您需要做什么。例如:
|
||||
- 「请调用插件,为我翻译PDF论文,论文我刚刚放到上传区了」
|
||||
@@ -104,9 +104,9 @@ def analyze_intention_with_simple_rules(txt):
|
||||
|
||||
|
||||
@CatchException
|
||||
def 虚空终端(txt, llm_kwargs, plugin_kwargs, chatbot, history, system_prompt, user_request):
|
||||
def Void_Terminal(txt, llm_kwargs, plugin_kwargs, chatbot, history, system_prompt, user_request):
|
||||
disable_auto_promotion(chatbot=chatbot)
|
||||
# 获取当前虚空终端状态
|
||||
# 获取当前Void_Terminal状态
|
||||
state = VoidTerminalState.get_state(chatbot)
|
||||
appendix_msg = ""
|
||||
|
||||
@@ -121,21 +121,21 @@ def 虚空终端(txt, llm_kwargs, plugin_kwargs, chatbot, history, system_prompt
|
||||
state.set_state(chatbot=chatbot, key='has_provided_explanation', value=True)
|
||||
state.unlock_plugin(chatbot=chatbot)
|
||||
yield from update_ui(chatbot=chatbot, history=history)
|
||||
yield from 虚空终端主路由(txt, llm_kwargs, plugin_kwargs, chatbot, history, system_prompt, user_request)
|
||||
yield from Void_Terminal主路由(txt, llm_kwargs, plugin_kwargs, chatbot, history, system_prompt, user_request)
|
||||
return
|
||||
else:
|
||||
# 如果意图模糊,提示
|
||||
state.set_state(chatbot=chatbot, key='has_provided_explanation', value=True)
|
||||
state.lock_plugin(chatbot=chatbot)
|
||||
chatbot.append(("虚空终端状态:", explain_msg+appendix_msg))
|
||||
chatbot.append(("Void_Terminal状态:", explain_msg+appendix_msg))
|
||||
yield from update_ui(chatbot=chatbot, history=history)
|
||||
return
|
||||
|
||||
|
||||
|
||||
def 虚空终端主路由(txt, llm_kwargs, plugin_kwargs, chatbot, history, system_prompt, user_request):
|
||||
def Void_Terminal主路由(txt, llm_kwargs, plugin_kwargs, chatbot, history, system_prompt, user_request):
|
||||
history = []
|
||||
chatbot.append(("虚空终端状态: ", f"正在执行任务: {txt}"))
|
||||
chatbot.append(("Void_Terminal状态: ", f"正在执行任务: {txt}"))
|
||||
yield from update_ui(chatbot=chatbot, history=history) # 刷新界面
|
||||
|
||||
# ⭐ ⭐ ⭐ 分析用户意图
|
||||
@@ -79,13 +79,13 @@ def 解析docx(file_manifest, project_folder, llm_kwargs, plugin_kwargs, chatbot
|
||||
|
||||
|
||||
@CatchException
|
||||
def 总结word文档(txt, llm_kwargs, plugin_kwargs, chatbot, history, system_prompt, user_request):
|
||||
def Word_Summary(txt, llm_kwargs, plugin_kwargs, chatbot, history, system_prompt, user_request):
|
||||
import glob, os
|
||||
|
||||
# 基本信息:功能、贡献者
|
||||
chatbot.append([
|
||||
"函数插件功能?",
|
||||
"批量总结Word文档。函数插件贡献者: JasonGuo1。注意, 如果是.doc文件, 请先转化为.docx格式。"])
|
||||
"批量Word_Summary。函数插件贡献者: JasonGuo1。注意, 如果是.doc文件, 请先转化为.docx格式。"])
|
||||
yield from update_ui(chatbot=chatbot, history=history) # 刷新界面
|
||||
|
||||
# 尝试导入依赖,如果缺少依赖,则给出安装建议
|
||||
@@ -9,7 +9,7 @@ from tqdm import tqdm
|
||||
|
||||
class ArxivSource(DataSource):
|
||||
"""arXiv API实现"""
|
||||
|
||||
|
||||
CATEGORIES = {
|
||||
# 物理学
|
||||
"Physics": {
|
||||
@@ -27,7 +27,7 @@ class ArxivSource(DataSource):
|
||||
"physics": "物理学",
|
||||
"quant-ph": "量子物理",
|
||||
},
|
||||
|
||||
|
||||
# 数学
|
||||
"Mathematics": {
|
||||
"math.AG": "代数几何",
|
||||
@@ -63,7 +63,7 @@ class ArxivSource(DataSource):
|
||||
"math.ST": "统计理论",
|
||||
"math.SG": "辛几何",
|
||||
},
|
||||
|
||||
|
||||
# 计算机科学
|
||||
"Computer Science": {
|
||||
"cs.AI": "人工智能",
|
||||
@@ -107,7 +107,7 @@ class ArxivSource(DataSource):
|
||||
"cs.SC": "符号计算",
|
||||
"cs.SY": "系统与控制",
|
||||
},
|
||||
|
||||
|
||||
# 定量生物学
|
||||
"Quantitative Biology": {
|
||||
"q-bio.BM": "生物分子",
|
||||
@@ -121,7 +121,7 @@ class ArxivSource(DataSource):
|
||||
"q-bio.SC": "亚细胞过程",
|
||||
"q-bio.TO": "组织与器官",
|
||||
},
|
||||
|
||||
|
||||
# 定量金融
|
||||
"Quantitative Finance": {
|
||||
"q-fin.CP": "计算金融",
|
||||
@@ -134,7 +134,7 @@ class ArxivSource(DataSource):
|
||||
"q-fin.ST": "统计金融",
|
||||
"q-fin.TR": "交易与市场微观结构",
|
||||
},
|
||||
|
||||
|
||||
# 统计学
|
||||
"Statistics": {
|
||||
"stat.AP": "应用统计",
|
||||
@@ -144,7 +144,7 @@ class ArxivSource(DataSource):
|
||||
"stat.OT": "其他统计",
|
||||
"stat.TH": "统计理论",
|
||||
},
|
||||
|
||||
|
||||
# 电气工程与系统科学
|
||||
"Electrical Engineering and Systems Science": {
|
||||
"eess.AS": "音频与语音处理",
|
||||
@@ -152,7 +152,7 @@ class ArxivSource(DataSource):
|
||||
"eess.SP": "信号处理",
|
||||
"eess.SY": "系统与控制",
|
||||
},
|
||||
|
||||
|
||||
# 经济学
|
||||
"Economics": {
|
||||
"econ.EM": "计量经济学",
|
||||
@@ -170,15 +170,15 @@ class ArxivSource(DataSource):
|
||||
'lastUpdatedDate': arxiv.SortCriterion.LastUpdatedDate, # 最后更新日期
|
||||
'submittedDate': arxiv.SortCriterion.SubmittedDate, # 提交日期
|
||||
}
|
||||
|
||||
|
||||
self.sort_order_options = {
|
||||
'ascending': arxiv.SortOrder.Ascending,
|
||||
'descending': arxiv.SortOrder.Descending
|
||||
}
|
||||
|
||||
|
||||
self.default_sort = 'lastUpdatedDate'
|
||||
self.default_order = 'descending'
|
||||
|
||||
|
||||
def _initialize(self) -> None:
|
||||
"""初始化客户端,设置默认参数"""
|
||||
self.client = arxiv.Client()
|
||||
@@ -196,22 +196,22 @@ class ArxivSource(DataSource):
|
||||
# 使用默认排序如果提供的排序选项无效
|
||||
if not sort_by or sort_by not in self.sort_options:
|
||||
sort_by = self.default_sort
|
||||
|
||||
# 使用默认排序顺序如果提供的顺序无效
|
||||
|
||||
# 使用默认排序顺序如果提供的顺序无效
|
||||
if not sort_order or sort_order not in self.sort_order_options:
|
||||
sort_order = self.default_order
|
||||
|
||||
|
||||
# 如果指定了起始年份,添加到查询中
|
||||
if start_year:
|
||||
query = f"{query} AND submittedDate:[{start_year}0101 TO 99991231]"
|
||||
|
||||
|
||||
search = arxiv.Search(
|
||||
query=query,
|
||||
max_results=limit,
|
||||
sort_by=self.sort_options[sort_by],
|
||||
sort_order=self.sort_order_options[sort_order]
|
||||
)
|
||||
|
||||
|
||||
results = list(self.client.results(search))
|
||||
return [self._parse_paper_data(result) for result in results]
|
||||
except Exception as e:
|
||||
@@ -220,13 +220,13 @@ class ArxivSource(DataSource):
|
||||
|
||||
async def search_by_id(self, paper_id: Union[str, List[str]]) -> List[PaperMetadata]:
|
||||
"""按ID搜索论文
|
||||
|
||||
|
||||
Args:
|
||||
paper_id: 单个arXiv ID或ID列表,例如:'2005.14165' 或 ['2005.14165', '2103.14030']
|
||||
"""
|
||||
if isinstance(paper_id, str):
|
||||
paper_id = [paper_id]
|
||||
|
||||
|
||||
search = arxiv.Search(
|
||||
id_list=paper_id,
|
||||
max_results=len(paper_id)
|
||||
@@ -235,8 +235,8 @@ class ArxivSource(DataSource):
|
||||
return [self._parse_paper_data(result) for result in results]
|
||||
|
||||
async def search_by_category(
|
||||
self,
|
||||
category: str,
|
||||
self,
|
||||
category: str,
|
||||
limit: int = 100,
|
||||
sort_by: str = 'relevance',
|
||||
sort_order: str = 'descending',
|
||||
@@ -244,11 +244,11 @@ class ArxivSource(DataSource):
|
||||
) -> List[PaperMetadata]:
|
||||
"""按类别搜索论文"""
|
||||
query = f"cat:{category}"
|
||||
|
||||
|
||||
# 如果指定了起始年份,添加到查询中
|
||||
if start_year:
|
||||
query = f"{query} AND submittedDate:[{start_year}0101 TO 99991231]"
|
||||
|
||||
|
||||
return await self.search(
|
||||
query=query,
|
||||
limit=limit,
|
||||
@@ -257,19 +257,19 @@ class ArxivSource(DataSource):
|
||||
)
|
||||
|
||||
async def search_by_authors(
|
||||
self,
|
||||
authors: List[str],
|
||||
self,
|
||||
authors: List[str],
|
||||
limit: int = 100,
|
||||
sort_by: str = 'relevance',
|
||||
start_year: int = None
|
||||
) -> List[PaperMetadata]:
|
||||
"""按作者搜索论文"""
|
||||
query = " AND ".join([f"au:\"{author}\"" for author in authors])
|
||||
|
||||
|
||||
# 如果指定了起始年份,添加到查询中
|
||||
if start_year:
|
||||
query = f"{query} AND submittedDate:[{start_year}0101 TO 99991231]"
|
||||
|
||||
|
||||
return await self.search(
|
||||
query=query,
|
||||
limit=limit,
|
||||
@@ -277,9 +277,9 @@ class ArxivSource(DataSource):
|
||||
)
|
||||
|
||||
async def search_by_date_range(
|
||||
self,
|
||||
start_date: datetime,
|
||||
end_date: datetime,
|
||||
self,
|
||||
start_date: datetime,
|
||||
end_date: datetime,
|
||||
limit: int = 100,
|
||||
sort_by: Literal['relevance', 'updated', 'submitted'] = 'submitted',
|
||||
sort_order: Literal['ascending', 'descending'] = 'descending'
|
||||
@@ -287,20 +287,20 @@ class ArxivSource(DataSource):
|
||||
"""按日期范围搜索论文"""
|
||||
query = f"submittedDate:[{start_date.strftime('%Y%m%d')} TO {end_date.strftime('%Y%m%d')}]"
|
||||
return await self.search(
|
||||
query,
|
||||
limit=limit,
|
||||
sort_by=sort_by,
|
||||
query,
|
||||
limit=limit,
|
||||
sort_by=sort_by,
|
||||
sort_order=sort_order
|
||||
)
|
||||
|
||||
async def download_pdf(self, paper_id: str, dirpath: str = "./", filename: str = "") -> str:
|
||||
"""下载论文PDF
|
||||
|
||||
|
||||
Args:
|
||||
paper_id: arXiv ID
|
||||
dirpath: 保存目录
|
||||
filename: 文件名,如果为空则使用默认格式:{paper_id}_{标题}.pdf
|
||||
|
||||
|
||||
Returns:
|
||||
保存的文件路径
|
||||
"""
|
||||
@@ -308,24 +308,24 @@ class ArxivSource(DataSource):
|
||||
if not papers:
|
||||
raise ValueError(f"未找到ID为 {paper_id} 的论文")
|
||||
paper = papers[0]
|
||||
|
||||
|
||||
if not filename:
|
||||
# 清理标题中的非法字符
|
||||
safe_title = "".join(c if c.isalnum() else "_" for c in paper.title)
|
||||
filename = f"{paper_id}_{safe_title}.pdf"
|
||||
|
||||
|
||||
filepath = os.path.join(dirpath, filename)
|
||||
urlretrieve(paper.url, filepath)
|
||||
return filepath
|
||||
|
||||
async def download_source(self, paper_id: str, dirpath: str = "./", filename: str = "") -> str:
|
||||
"""下载论文源文件(通常是LaTeX源码)
|
||||
|
||||
|
||||
Args:
|
||||
paper_id: arXiv ID
|
||||
dirpath: 保存目录
|
||||
filename: 文件名,如果为空则使用默认格式:{paper_id}_{标题}.tar.gz
|
||||
|
||||
|
||||
Returns:
|
||||
保存的文件路径
|
||||
"""
|
||||
@@ -333,11 +333,11 @@ class ArxivSource(DataSource):
|
||||
if not papers:
|
||||
raise ValueError(f"未找到ID为 {paper_id} 的论文")
|
||||
paper = papers[0]
|
||||
|
||||
|
||||
if not filename:
|
||||
safe_title = "".join(c if c.isalnum() else "_" for c in paper.title)
|
||||
filename = f"{paper_id}_{safe_title}.tar.gz"
|
||||
|
||||
|
||||
filepath = os.path.join(dirpath, filename)
|
||||
source_url = paper.url.replace("/pdf/", "/src/")
|
||||
urlretrieve(source_url, filepath)
|
||||
@@ -353,10 +353,10 @@ class ArxivSource(DataSource):
|
||||
|
||||
async def get_paper_details(self, paper_id: str) -> Optional[PaperMetadata]:
|
||||
"""获取论文详情
|
||||
|
||||
|
||||
Args:
|
||||
paper_id: arXiv ID 或 DOI
|
||||
|
||||
|
||||
Returns:
|
||||
论文详细信息,如果未找到返回 None
|
||||
"""
|
||||
@@ -367,7 +367,7 @@ class ArxivSource(DataSource):
|
||||
# 如果是 DOI 格式且是 arXiv 论文,提取 ID
|
||||
elif paper_id.startswith("10.48550/arXiv."):
|
||||
paper_id = paper_id.split(".")[-1]
|
||||
|
||||
|
||||
papers = await self.search_by_id(paper_id)
|
||||
return papers[0] if papers else None
|
||||
except Exception as e:
|
||||
@@ -379,7 +379,7 @@ class ArxivSource(DataSource):
|
||||
# 解析主要类别和次要类别
|
||||
primary_category = result.primary_category
|
||||
categories = result.categories
|
||||
|
||||
|
||||
# 构建venue信息
|
||||
venue_info = {
|
||||
'primary_category': primary_category,
|
||||
@@ -387,7 +387,7 @@ class ArxivSource(DataSource):
|
||||
'comments': getattr(result, 'comment', None),
|
||||
'journal_ref': getattr(result, 'journal_ref', None)
|
||||
}
|
||||
|
||||
|
||||
return PaperMetadata(
|
||||
title=result.title,
|
||||
authors=[author.name for author in result.authors],
|
||||
@@ -405,15 +405,15 @@ class ArxivSource(DataSource):
|
||||
)
|
||||
|
||||
async def get_latest_papers(
|
||||
self,
|
||||
category: str,
|
||||
self,
|
||||
category: str,
|
||||
debug: bool = False,
|
||||
batch_size: int = 50
|
||||
) -> List[PaperMetadata]:
|
||||
"""获取指定类别的最新论文
|
||||
|
||||
|
||||
通过 RSS feed 获取最新发布的论文,然后批量获取详细信息
|
||||
|
||||
|
||||
Args:
|
||||
category: arXiv类别,例如:
|
||||
- 整个领域: 'cs'
|
||||
@@ -421,10 +421,10 @@ class ArxivSource(DataSource):
|
||||
- 多个类别: 'cs.AI+q-bio.NC'
|
||||
debug: 是否为调试模式,如果为True则只返回5篇最新论文
|
||||
batch_size: 批量获取论文的数量,默认50
|
||||
|
||||
|
||||
Returns:
|
||||
论文列表
|
||||
|
||||
|
||||
Raises:
|
||||
ValueError: 如果类别无效
|
||||
"""
|
||||
@@ -433,22 +433,22 @@ class ArxivSource(DataSource):
|
||||
# 1. 转换为小写
|
||||
# 2. 确保多个类别之间使用+连接
|
||||
category = category.lower().replace(' ', '+')
|
||||
|
||||
|
||||
# 构建RSS feed URL
|
||||
feed_url = f"https://rss.arxiv.org/rss/{category}"
|
||||
print(f"正在获取RSS feed: {feed_url}") # 添加调试信息
|
||||
|
||||
|
||||
feed = feedparser.parse(feed_url)
|
||||
|
||||
|
||||
# 检查feed是否有效
|
||||
if hasattr(feed, 'status') and feed.status != 200:
|
||||
raise ValueError(f"获取RSS feed失败,状态码: {feed.status}")
|
||||
|
||||
|
||||
if not feed.entries:
|
||||
print(f"警告:未在feed中找到任何条目") # 添加调试信息
|
||||
print(f"Feed标题: {feed.feed.title if hasattr(feed, 'feed') else '无标题'}")
|
||||
raise ValueError(f"无效的arXiv类别或未找到论文: {category}")
|
||||
|
||||
|
||||
if debug:
|
||||
# 调试模式:只获取5篇最新论文
|
||||
search = arxiv.Search(
|
||||
@@ -459,7 +459,7 @@ class ArxivSource(DataSource):
|
||||
)
|
||||
results = list(self.client.results(search))
|
||||
return [self._parse_paper_data(result) for result in results]
|
||||
|
||||
|
||||
# 正常模式:获取所有新论文
|
||||
# 从RSS条目中提取arXiv ID
|
||||
paper_ids = []
|
||||
@@ -476,13 +476,13 @@ class ArxivSource(DataSource):
|
||||
except Exception as e:
|
||||
print(f"警告:处理条目时出错: {str(e)}") # 添加调试信息
|
||||
continue
|
||||
|
||||
|
||||
if not paper_ids:
|
||||
print("未能从feed中提取到任何论文ID") # 添加调试信息
|
||||
return []
|
||||
|
||||
|
||||
print(f"成功提取到 {len(paper_ids)} 个论文ID") # 添加调试信息
|
||||
|
||||
|
||||
# 批量获取论文详情
|
||||
papers = []
|
||||
with tqdm(total=len(paper_ids), desc="获取arXiv论文") as pbar:
|
||||
@@ -495,9 +495,9 @@ class ArxivSource(DataSource):
|
||||
batch_results = list(self.client.results(search))
|
||||
papers.extend([self._parse_paper_data(result) for result in batch_results])
|
||||
pbar.update(len(batch_results))
|
||||
|
||||
|
||||
return papers
|
||||
|
||||
|
||||
except Exception as e:
|
||||
print(f"获取最新论文时发生错误: {str(e)}")
|
||||
import traceback
|
||||
@@ -507,18 +507,18 @@ class ArxivSource(DataSource):
|
||||
async def example_usage():
|
||||
"""ArxivSource使用示例"""
|
||||
arxiv_source = ArxivSource()
|
||||
|
||||
|
||||
try:
|
||||
# 示例1:基本搜索,使用不同的排序方式
|
||||
# print("\n=== 示例1:搜索最新的机器学习论文(按提交时间排序)===")
|
||||
# papers = await arxiv_source.search(
|
||||
# "ti:\"machine learning\"",
|
||||
# "ti:\"machine learning\"",
|
||||
# limit=3,
|
||||
# sort_by='submitted',
|
||||
# sort_order='descending'
|
||||
# )
|
||||
# print(f"找到 {len(papers)} 篇论文")
|
||||
|
||||
|
||||
# for i, paper in enumerate(papers, 1):
|
||||
# print(f"\n--- 论文 {i} ---")
|
||||
# print(f"标题: {paper.title}")
|
||||
@@ -544,7 +544,7 @@ async def example_usage():
|
||||
# # 示例3:按类别搜索
|
||||
# print("\n=== 示例3:搜索人工智能领域最新论文 ===")
|
||||
# ai_papers = await arxiv_source.search_by_category(
|
||||
# "cs.AI",
|
||||
# "cs.AI",
|
||||
# limit=2,
|
||||
# sort_by='updated',
|
||||
# sort_order='descending'
|
||||
@@ -558,7 +558,7 @@ async def example_usage():
|
||||
# # 示例4:按作者搜索
|
||||
# print("\n=== 示例4:搜索特定作者的论文 ===")
|
||||
# author_papers = await arxiv_source.search_by_authors(
|
||||
# ["Bengio"],
|
||||
# ["Bengio"],
|
||||
# limit=2,
|
||||
# sort_by='relevance'
|
||||
# )
|
||||
@@ -598,7 +598,7 @@ async def example_usage():
|
||||
|
||||
# 示例6:获取最新论文
|
||||
print("\n=== 示例8:获取最新论文 ===")
|
||||
|
||||
|
||||
# 获取CS.AI领域的最新论文
|
||||
print("\n--- 获取AI领域最新论文 ---")
|
||||
ai_latest = await arxiv_source.get_latest_papers("cs.AI", debug=True)
|
||||
@@ -607,7 +607,7 @@ async def example_usage():
|
||||
print(f"标题: {paper.title}")
|
||||
print(f"作者: {', '.join(paper.authors)}")
|
||||
print(f"发表年份: {paper.year}")
|
||||
|
||||
|
||||
# 获取整个计算机科学领域的最新论文
|
||||
print("\n--- 获取整个CS领域最新论文 ---")
|
||||
cs_latest = await arxiv_source.get_latest_papers("cs", debug=True)
|
||||
@@ -616,7 +616,7 @@ async def example_usage():
|
||||
print(f"标题: {paper.title}")
|
||||
print(f"作者: {', '.join(paper.authors)}")
|
||||
print(f"发表年份: {paper.year}")
|
||||
|
||||
|
||||
# 获取多个类别的最新论文
|
||||
print("\n--- 获取AI和机器学习领域最新论文 ---")
|
||||
multi_latest = await arxiv_source.get_latest_papers("cs.AI+cs.LG", debug=True)
|
||||
@@ -633,4 +633,4 @@ async def example_usage():
|
||||
|
||||
if __name__ == "__main__":
|
||||
import asyncio
|
||||
asyncio.run(example_usage())
|
||||
asyncio.run(example_usage())
|
||||
@@ -8,7 +8,7 @@ class VoidTerminalState():
|
||||
self.has_provided_explanation = False
|
||||
|
||||
def lock_plugin(self, chatbot):
|
||||
chatbot._cookies['lock_plugin'] = 'crazy_functions.虚空终端->虚空终端'
|
||||
chatbot._cookies['lock_plugin'] = 'crazy_functions.Void_Terminal->Void_Terminal'
|
||||
chatbot._cookies['plugin_state'] = pickle.dumps(self)
|
||||
|
||||
def unlock_plugin(self, chatbot):
|
||||
|
||||
@@ -1,162 +0,0 @@
|
||||
from loguru import logger
|
||||
from toolbox import update_ui
|
||||
from toolbox import CatchException, report_exception
|
||||
from crazy_functions.crazy_utils import request_gpt_model_in_new_thread_with_ui_alive
|
||||
from toolbox import write_history_to_file, promote_file_to_downloadzone
|
||||
|
||||
fast_debug = False
|
||||
|
||||
def readPdf(pdfPath):
|
||||
"""
|
||||
读取pdf文件,返回文本内容
|
||||
"""
|
||||
import pdfminer
|
||||
from pdfminer.pdfparser import PDFParser
|
||||
from pdfminer.pdfdocument import PDFDocument
|
||||
from pdfminer.pdfpage import PDFPage, PDFTextExtractionNotAllowed
|
||||
from pdfminer.pdfinterp import PDFResourceManager, PDFPageInterpreter
|
||||
from pdfminer.pdfdevice import PDFDevice
|
||||
from pdfminer.layout import LAParams
|
||||
from pdfminer.converter import PDFPageAggregator
|
||||
|
||||
fp = open(pdfPath, 'rb')
|
||||
|
||||
# Create a PDF parser object associated with the file object
|
||||
parser = PDFParser(fp)
|
||||
|
||||
# Create a PDF document object that stores the document structure.
|
||||
# Password for initialization as 2nd parameter
|
||||
document = PDFDocument(parser)
|
||||
# Check if the document allows text extraction. If not, abort.
|
||||
if not document.is_extractable:
|
||||
raise PDFTextExtractionNotAllowed
|
||||
|
||||
# Create a PDF resource manager object that stores shared resources.
|
||||
rsrcmgr = PDFResourceManager()
|
||||
|
||||
# Create a PDF device object.
|
||||
# device = PDFDevice(rsrcmgr)
|
||||
|
||||
# BEGIN LAYOUT ANALYSIS.
|
||||
# Set parameters for analysis.
|
||||
laparams = LAParams(
|
||||
char_margin=10.0,
|
||||
line_margin=0.2,
|
||||
boxes_flow=0.2,
|
||||
all_texts=False,
|
||||
)
|
||||
# Create a PDF page aggregator object.
|
||||
device = PDFPageAggregator(rsrcmgr, laparams=laparams)
|
||||
# Create a PDF interpreter object.
|
||||
interpreter = PDFPageInterpreter(rsrcmgr, device)
|
||||
|
||||
# loop over all pages in the document
|
||||
outTextList = []
|
||||
for page in PDFPage.create_pages(document):
|
||||
# read the page into a layout object
|
||||
interpreter.process_page(page)
|
||||
layout = device.get_result()
|
||||
for obj in layout._objs:
|
||||
if isinstance(obj, pdfminer.layout.LTTextBoxHorizontal):
|
||||
outTextList.append(obj.get_text())
|
||||
|
||||
return outTextList
|
||||
|
||||
|
||||
def 解析Paper(file_manifest, project_folder, llm_kwargs, plugin_kwargs, chatbot, history, system_prompt):
|
||||
import time, glob, os
|
||||
from bs4 import BeautifulSoup
|
||||
logger.info('begin analysis on:', file_manifest)
|
||||
for index, fp in enumerate(file_manifest):
|
||||
if ".tex" in fp:
|
||||
with open(fp, 'r', encoding='utf-8', errors='replace') as f:
|
||||
file_content = f.read()
|
||||
if ".pdf" in fp.lower():
|
||||
file_content = readPdf(fp)
|
||||
file_content = BeautifulSoup(''.join(file_content), features="lxml").body.text.encode('gbk', 'ignore').decode('gbk')
|
||||
|
||||
prefix = "接下来请你逐文件分析下面的论文文件,概括其内容" if index==0 else ""
|
||||
i_say = prefix + f'请对下面的文章片段用中文做一个概述,文件名是{os.path.relpath(fp, project_folder)},文章内容是 ```{file_content}```'
|
||||
i_say_show_user = prefix + f'[{index+1}/{len(file_manifest)}] 请对下面的文章片段做一个概述: {os.path.abspath(fp)}'
|
||||
chatbot.append((i_say_show_user, "[Local Message] waiting gpt response."))
|
||||
yield from update_ui(chatbot=chatbot, history=history) # 刷新界面
|
||||
|
||||
if not fast_debug:
|
||||
msg = '正常'
|
||||
# ** gpt request **
|
||||
gpt_say = yield from request_gpt_model_in_new_thread_with_ui_alive(
|
||||
inputs=i_say,
|
||||
inputs_show_user=i_say_show_user,
|
||||
llm_kwargs=llm_kwargs,
|
||||
chatbot=chatbot,
|
||||
history=[],
|
||||
sys_prompt="总结文章。"
|
||||
) # 带超时倒计时
|
||||
chatbot[-1] = (i_say_show_user, gpt_say)
|
||||
history.append(i_say_show_user); history.append(gpt_say)
|
||||
yield from update_ui(chatbot=chatbot, history=history, msg=msg) # 刷新界面
|
||||
if not fast_debug: time.sleep(2)
|
||||
|
||||
all_file = ', '.join([os.path.relpath(fp, project_folder) for index, fp in enumerate(file_manifest)])
|
||||
i_say = f'根据以上你自己的分析,对全文进行概括,用学术性语言写一段中文摘要,然后再写一段英文摘要(包括{all_file})。'
|
||||
chatbot.append((i_say, "[Local Message] waiting gpt response."))
|
||||
yield from update_ui(chatbot=chatbot, history=history) # 刷新界面
|
||||
|
||||
if not fast_debug:
|
||||
msg = '正常'
|
||||
# ** gpt request **
|
||||
gpt_say = yield from request_gpt_model_in_new_thread_with_ui_alive(
|
||||
inputs=i_say,
|
||||
inputs_show_user=i_say,
|
||||
llm_kwargs=llm_kwargs,
|
||||
chatbot=chatbot,
|
||||
history=history,
|
||||
sys_prompt="总结文章。"
|
||||
) # 带超时倒计时
|
||||
chatbot[-1] = (i_say, gpt_say)
|
||||
history.append(i_say); history.append(gpt_say)
|
||||
yield from update_ui(chatbot=chatbot, history=history, msg=msg) # 刷新界面
|
||||
res = write_history_to_file(history)
|
||||
promote_file_to_downloadzone(res, chatbot=chatbot)
|
||||
chatbot.append(("完成了吗?", res))
|
||||
yield from update_ui(chatbot=chatbot, history=history, msg=msg) # 刷新界面
|
||||
|
||||
|
||||
|
||||
@CatchException
|
||||
def 批量总结PDF文档pdfminer(txt, llm_kwargs, plugin_kwargs, chatbot, history, system_prompt, user_request):
|
||||
history = [] # 清空历史,以免输入溢出
|
||||
import glob, os
|
||||
|
||||
# 基本信息:功能、贡献者
|
||||
chatbot.append([
|
||||
"函数插件功能?",
|
||||
"批量总结PDF文档,此版本使用pdfminer插件,带token约简功能。函数插件贡献者: Euclid-Jie。"])
|
||||
yield from update_ui(chatbot=chatbot, history=history) # 刷新界面
|
||||
|
||||
# 尝试导入依赖,如果缺少依赖,则给出安装建议
|
||||
try:
|
||||
import pdfminer, bs4
|
||||
except:
|
||||
report_exception(chatbot, history,
|
||||
a = f"解析项目: {txt}",
|
||||
b = f"导入软件依赖失败。使用该模块需要额外依赖,安装方法```pip install --upgrade pdfminer beautifulsoup4```。")
|
||||
yield from update_ui(chatbot=chatbot, history=history) # 刷新界面
|
||||
return
|
||||
if os.path.exists(txt):
|
||||
project_folder = txt
|
||||
else:
|
||||
if txt == "": txt = '空空如也的输入栏'
|
||||
report_exception(chatbot, history, a = f"解析项目: {txt}", b = f"找不到本地项目或无权访问: {txt}")
|
||||
yield from update_ui(chatbot=chatbot, history=history) # 刷新界面
|
||||
return
|
||||
file_manifest = [f for f in glob.glob(f'{project_folder}/**/*.tex', recursive=True)] + \
|
||||
[f for f in glob.glob(f'{project_folder}/**/*.pdf', recursive=True)] # + \
|
||||
# [f for f in glob.glob(f'{project_folder}/**/*.cpp', recursive=True)] + \
|
||||
# [f for f in glob.glob(f'{project_folder}/**/*.c', recursive=True)]
|
||||
if len(file_manifest) == 0:
|
||||
report_exception(chatbot, history, a = f"解析项目: {txt}", b = f"找不到任何.tex或pdf文件: {txt}")
|
||||
yield from update_ui(chatbot=chatbot, history=history) # 刷新界面
|
||||
return
|
||||
yield from 解析Paper(file_manifest, project_folder, llm_kwargs, plugin_kwargs, chatbot, history, system_prompt)
|
||||
|
||||
@@ -1 +0,0 @@
|
||||
# 此Dockerfile不再维护,请前往docs/GithubAction+ChatGLM+Moss
|
||||
@@ -1 +0,0 @@
|
||||
# 此Dockerfile不再维护,请前往docs/GithubAction+NoLocal+Latex
|
||||
@@ -1,43 +1,56 @@
|
||||
# docker build -t gpt-academic-all-capacity -f docs/GithubAction+AllCapacity --network=host --build-arg http_proxy=http://localhost:10881 --build-arg https_proxy=http://localhost:10881 .
|
||||
# 此Dockerfile适用于“无本地模型”的迷你运行环境构建
|
||||
# 如果需要使用chatglm等本地模型或者latex运行依赖,请参考 docker-compose.yml
|
||||
# - 如何构建: 先修改 `config.py`, 然后 `docker build -t gpt-academic . `
|
||||
# - 如何运行(Linux下): `docker run --rm -it --net=host gpt-academic `
|
||||
# - 如何运行(其他操作系统,选择任意一个固定端口50923): `docker run --rm -it -e WEB_PORT=50923 -p 50923:50923 gpt-academic `
|
||||
|
||||
# 从NVIDIA源,从而支持显卡(检查宿主的nvidia-smi中的cuda版本必须>=11.3)
|
||||
FROM fuqingxu/11.3.1-runtime-ubuntu20.04-with-texlive:latest
|
||||
|
||||
# edge-tts需要的依赖,某些pip包所需的依赖
|
||||
RUN apt update && apt install ffmpeg build-essential -y
|
||||
RUN apt-get install -y fontconfig
|
||||
FROM fuqingxu/11.3.1-runtime-ubuntu20.04-with-texlive:latest AS texlive
|
||||
FROM ghcr.io/astral-sh/uv:python3.12-bookworm
|
||||
|
||||
|
||||
# 非必要步骤,更换pip源 (以下三行,可以删除)
|
||||
RUN echo '[global]' > /etc/pip.conf && \
|
||||
echo 'index-url = https://mirrors.aliyun.com/pypi/simple/' >> /etc/pip.conf && \
|
||||
echo 'trusted-host = mirrors.aliyun.com' >> /etc/pip.conf
|
||||
|
||||
# 语音输出功能(以下1,2行更换阿里源,第3,4行安装ffmpeg,都可以删除)
|
||||
RUN sed -i 's/deb.debian.org/mirrors.aliyun.com/g' /etc/apt/sources.list.d/debian.sources && \
|
||||
sed -i 's/security.debian.org/mirrors.aliyun.com/g' /etc/apt/sources.list.d/debian.sources && \
|
||||
apt-get update
|
||||
RUN apt-get install ffmpeg fontconfig build-essential -y
|
||||
|
||||
RUN ln -s /usr/local/texlive/2023/texmf-dist/fonts/truetype /usr/share/fonts/truetype/texlive
|
||||
RUN fc-cache -fv
|
||||
RUN apt-get clean
|
||||
COPY --from=texlive /usr/local/texlive /usr/local/texlive
|
||||
|
||||
# use python3 as the system default python
|
||||
# 进入工作路径(必要)
|
||||
WORKDIR /gpt
|
||||
RUN curl -sS https://bootstrap.pypa.io/get-pip.py | python3.8
|
||||
# 下载pytorch
|
||||
RUN python3 -m pip install torch --extra-index-url https://download.pytorch.org/whl/cu113
|
||||
# 准备pip依赖
|
||||
RUN python3 -m pip install openai numpy arxiv rich
|
||||
RUN python3 -m pip install colorama Markdown pygments pymupdf
|
||||
RUN python3 -m pip install python-docx moviepy pdfminer
|
||||
RUN python3 -m pip install zh_langchain==0.2.1 pypinyin
|
||||
RUN python3 -m pip install rarfile py7zr
|
||||
RUN python3 -m pip install aliyun-python-sdk-core==2.13.3 pyOpenSSL webrtcvad scipy git+https://github.com/aliyun/alibabacloud-nls-python-sdk.git
|
||||
# 下载分支
|
||||
WORKDIR /gpt
|
||||
RUN git clone --depth=1 https://github.com/binary-husky/gpt_academic.git
|
||||
WORKDIR /gpt/gpt_academic
|
||||
RUN git clone --depth=1 https://github.com/OpenLMLab/MOSS.git request_llms/moss
|
||||
|
||||
RUN python3 -m pip install -r requirements.txt
|
||||
RUN python3 -m pip install -r request_llms/requirements_moss.txt
|
||||
RUN python3 -m pip install -r request_llms/requirements_qwen.txt
|
||||
RUN python3 -m pip install -r request_llms/requirements_chatglm.txt
|
||||
RUN python3 -m pip install -r request_llms/requirements_newbing.txt
|
||||
RUN python3 -m pip install nougat-ocr
|
||||
RUN python3 -m pip cache purge
|
||||
# 安装大部分依赖,利用缓存加速以后的构建
|
||||
COPY requirements.txt ./
|
||||
RUN uv venv --python=3.12
|
||||
RUN uv pip install --verbose -r requirements.txt -i https://mirrors.aliyun.com/pypi/simple/
|
||||
RUN uv pip install torch --verbose --index-url https://download.pytorch.org/whl/cpu
|
||||
RUN uv pip install openai numpy arxiv rich colorama Markdown pygments pymupdf python-docx moviepy pdfminer zh_langchain==0.2.1 pypinyin rarfile py7zr nougat-ocr -i https://mirrors.aliyun.com/pypi/simple/
|
||||
RUN uv pip install aliyun-python-sdk-core==2.13.3 pyOpenSSL webrtcvad scipy -i https://mirrors.aliyun.com/pypi/simple/
|
||||
RUN uv pip install alibabacloud-nls-python-sdk -i https://mirrors.aliyun.com/pypi/simple/
|
||||
ENV PATH="/gpt/.venv/bin:$PATH"
|
||||
|
||||
# 预热Tiktoken模块
|
||||
RUN python3 -c 'from check_proxy import warm_up_modules; warm_up_modules()'
|
||||
# 装载项目文件,安装剩余依赖(必要)
|
||||
COPY . .
|
||||
|
||||
# 启动
|
||||
CMD ["python3", "-u", "main.py"]
|
||||
RUN uv pip install -r requirements.txt -i https://mirrors.aliyun.com/pypi/simple/
|
||||
|
||||
# # 非必要步骤,用于预热模块(可以删除)
|
||||
RUN python -c 'from check_proxy import warm_up_modules; warm_up_modules()'
|
||||
|
||||
ENV CGO_ENABLED=0
|
||||
ENV PATH "/usr/local/texlive/2023/bin/x86_64-linux:$PATH"
|
||||
ENV PATH "/usr/local/texlive/2024/bin/x86_64-linux:$PATH"
|
||||
ENV PATH "/usr/local/texlive/2025/bin/x86_64-linux:$PATH"
|
||||
ENV PATH "/usr/local/texlive/2026/bin/x86_64-linux:$PATH"
|
||||
|
||||
# 启动(必要)
|
||||
CMD ["bash", "-c", "python main.py"]
|
||||
|
||||
@@ -1,24 +1,37 @@
|
||||
# 此Dockerfile适用于“无本地模型”的环境构建,如果需要使用chatglm等本地模型,请参考 docs/Dockerfile+ChatGLM
|
||||
# 如何构建: 先修改 `config.py`, 然后 docker build -t gpt-academic-nolocal -f docs/Dockerfile+NoLocal .
|
||||
# 如何运行: docker run --rm -it --net=host gpt-academic-nolocal
|
||||
FROM python:3.11
|
||||
FROM ghcr.io/astral-sh/uv:python3.12-bookworm
|
||||
|
||||
# 指定路径
|
||||
# 非必要步骤,更换pip源 (以下三行,可以删除)
|
||||
RUN echo '[global]' > /etc/pip.conf && \
|
||||
echo 'index-url = https://mirrors.aliyun.com/pypi/simple/' >> /etc/pip.conf && \
|
||||
echo 'trusted-host = mirrors.aliyun.com' >> /etc/pip.conf
|
||||
|
||||
# 语音输出功能(以下1,2行更换阿里源,第3,4行安装ffmpeg,都可以删除)
|
||||
RUN sed -i 's/deb.debian.org/mirrors.aliyun.com/g' /etc/apt/sources.list.d/debian.sources && \
|
||||
sed -i 's/security.debian.org/mirrors.aliyun.com/g' /etc/apt/sources.list.d/debian.sources && \
|
||||
apt-get update
|
||||
RUN apt-get install ffmpeg -y
|
||||
RUN apt-get clean
|
||||
|
||||
# 进入工作路径(必要)
|
||||
WORKDIR /gpt
|
||||
|
||||
# 装载项目文件
|
||||
# 安装大部分依赖,利用Docker缓存加速以后的构建 (以下两行,可以删除)
|
||||
COPY requirements.txt ./
|
||||
RUN uv venv --python=3.12 && uv pip install --verbose -r requirements.txt -i https://mirrors.aliyun.com/pypi/simple/
|
||||
ENV PATH="/gpt/.venv/bin:$PATH"
|
||||
RUN python -c 'import loguru'
|
||||
|
||||
# 装载项目文件,安装剩余依赖(必要)
|
||||
COPY . .
|
||||
RUN uv pip install -r requirements.txt -i https://mirrors.aliyun.com/pypi/simple/
|
||||
|
||||
# 安装依赖
|
||||
RUN pip3 install -r requirements.txt
|
||||
# # 非必要步骤,用于预热模块(可以删除)
|
||||
RUN python -c 'from check_proxy import warm_up_modules; warm_up_modules()'
|
||||
|
||||
# edge-tts需要的依赖
|
||||
RUN apt update && apt install ffmpeg -y
|
||||
ENV CGO_ENABLED=0
|
||||
|
||||
# 可选步骤,用于预热模块
|
||||
RUN python3 -c 'from check_proxy import warm_up_modules; warm_up_modules()'
|
||||
|
||||
RUN python3 -m pip cache purge && apt-get clean
|
||||
|
||||
# 启动
|
||||
CMD ["python3", "-u", "main.py"]
|
||||
# 启动(必要)
|
||||
CMD ["bash", "-c", "python main.py"]
|
||||
@@ -1,25 +1,43 @@
|
||||
# 此Dockerfile适用于“无本地模型”的环境构建,如果需要使用chatglm等本地模型,请参考 docs/Dockerfile+ChatGLM
|
||||
# 如何构建: 先修改 `config.py`, 然后 docker build -t gpt-academic-nolocal -f docs/Dockerfile+NoLocal .
|
||||
# 如何运行: docker run --rm -it --net=host gpt-academic-nolocal
|
||||
FROM python:3.11
|
||||
# 此Dockerfile适用于“无本地模型”的迷你运行环境构建
|
||||
# 如果需要使用chatglm等本地模型或者latex运行依赖,请参考 docker-compose.yml
|
||||
# - 如何构建: 先修改 `config.py`, 然后 `docker build -t gpt-academic . `
|
||||
# - 如何运行(Linux下): `docker run --rm -it --net=host gpt-academic `
|
||||
# - 如何运行(其他操作系统,选择任意一个固定端口50923): `docker run --rm -it -e WEB_PORT=50923 -p 50923:50923 gpt-academic `
|
||||
|
||||
# 指定路径
|
||||
FROM ghcr.io/astral-sh/uv:python3.12-bookworm
|
||||
|
||||
# 非必要步骤,更换pip源 (以下三行,可以删除)
|
||||
RUN echo '[global]' > /etc/pip.conf && \
|
||||
echo 'index-url = https://mirrors.aliyun.com/pypi/simple/' >> /etc/pip.conf && \
|
||||
echo 'trusted-host = mirrors.aliyun.com' >> /etc/pip.conf
|
||||
|
||||
# 语音输出功能(以下1,2行更换阿里源,第3,4行安装ffmpeg,都可以删除)
|
||||
RUN sed -i 's/deb.debian.org/mirrors.aliyun.com/g' /etc/apt/sources.list.d/debian.sources && \
|
||||
sed -i 's/security.debian.org/mirrors.aliyun.com/g' /etc/apt/sources.list.d/debian.sources && \
|
||||
apt-get update
|
||||
RUN apt-get install ffmpeg -y
|
||||
RUN apt-get clean
|
||||
|
||||
# 进入工作路径(必要)
|
||||
WORKDIR /gpt
|
||||
|
||||
# 装载项目文件
|
||||
# 安装大部分依赖,利用Docker缓存加速以后的构建 (以下两行,可以删除)
|
||||
COPY requirements.txt ./
|
||||
RUN uv venv --python=3.12 && uv pip install --verbose -r requirements.txt -i https://mirrors.aliyun.com/pypi/simple/
|
||||
RUN uv pip install aliyun-python-sdk-core==2.13.3 pyOpenSSL webrtcvad scipy -i https://mirrors.aliyun.com/pypi/simple/
|
||||
RUN uv pip install alibabacloud-nls-python-sdk -i https://mirrors.aliyun.com/pypi/simple/
|
||||
|
||||
ENV PATH="/gpt/.venv/bin:$PATH"
|
||||
RUN python -c 'import loguru'
|
||||
|
||||
# 装载项目文件,安装剩余依赖(必要)
|
||||
COPY . .
|
||||
RUN uv pip install -r requirements.txt -i https://mirrors.aliyun.com/pypi/simple/
|
||||
|
||||
# 安装依赖
|
||||
RUN pip3 install -r requirements.txt
|
||||
# # 非必要步骤,用于预热模块(可以删除)
|
||||
RUN python -c 'from check_proxy import warm_up_modules; warm_up_modules()'
|
||||
|
||||
# 安装语音插件的额外依赖
|
||||
RUN pip3 install aliyun-python-sdk-core==2.13.3 pyOpenSSL webrtcvad scipy git+https://github.com/aliyun/alibabacloud-nls-python-sdk.git
|
||||
ENV CGO_ENABLED=0
|
||||
|
||||
# edge-tts需要的依赖
|
||||
RUN apt update && apt install ffmpeg -y
|
||||
|
||||
# 可选步骤,用于预热模块
|
||||
RUN python3 -c 'from check_proxy import warm_up_modules; warm_up_modules()'
|
||||
|
||||
# 启动
|
||||
CMD ["python3", "-u", "main.py"]
|
||||
# 启动(必要)
|
||||
CMD ["bash", "-c", "python main.py"]
|
||||
@@ -1,31 +1,48 @@
|
||||
# 此Dockerfile适用于“无本地模型”的环境构建,如果需要使用chatglm等本地模型,请参考 docs/Dockerfile+ChatGLM
|
||||
# 如何构建: 先修改 `config.py`, 然后 docker build -t gpt-academic-nolocal-vs -f docs/GithubAction+NoLocal+Vectordb .
|
||||
# 如何运行: docker run --rm -it --net=host gpt-academic-nolocal-vs
|
||||
FROM python:3.11
|
||||
# 此Dockerfile适用于“无本地模型”的迷你运行环境构建
|
||||
# 如果需要使用chatglm等本地模型或者latex运行依赖,请参考 docker-compose.yml
|
||||
# - 如何构建: 先修改 `config.py`, 然后 `docker build -t gpt-academic . `
|
||||
# - 如何运行(Linux下): `docker run --rm -it --net=host gpt-academic `
|
||||
# - 如何运行(其他操作系统,选择任意一个固定端口50923): `docker run --rm -it -e WEB_PORT=50923 -p 50923:50923 gpt-academic `
|
||||
|
||||
# 指定路径
|
||||
FROM ghcr.io/astral-sh/uv:python3.12-bookworm
|
||||
|
||||
# 非必要步骤,更换pip源 (以下三行,可以删除)
|
||||
RUN echo '[global]' > /etc/pip.conf && \
|
||||
echo 'index-url = https://mirrors.aliyun.com/pypi/simple/' >> /etc/pip.conf && \
|
||||
echo 'trusted-host = mirrors.aliyun.com' >> /etc/pip.conf
|
||||
|
||||
# 语音输出功能(以下1,2行更换阿里源,第3,4行安装ffmpeg,都可以删除)
|
||||
RUN sed -i 's/deb.debian.org/mirrors.aliyun.com/g' /etc/apt/sources.list.d/debian.sources && \
|
||||
sed -i 's/security.debian.org/mirrors.aliyun.com/g' /etc/apt/sources.list.d/debian.sources && \
|
||||
apt-get update
|
||||
RUN apt-get install ffmpeg -y
|
||||
RUN apt-get clean
|
||||
|
||||
# 进入工作路径(必要)
|
||||
WORKDIR /gpt
|
||||
|
||||
# 装载项目文件
|
||||
# 安装大部分依赖,利用Docker缓存加速以后的构建 (以下两行,可以删除)
|
||||
COPY requirements.txt ./
|
||||
RUN uv venv --python=3.12 && uv pip install --verbose -r requirements.txt -i https://mirrors.aliyun.com/pypi/simple/
|
||||
RUN uv pip install aliyun-python-sdk-core==2.13.3 pyOpenSSL webrtcvad scipy -i https://mirrors.aliyun.com/pypi/simple/
|
||||
RUN uv pip install alibabacloud-nls-python-sdk -i https://mirrors.aliyun.com/pypi/simple/
|
||||
RUN uv pip install torch torchvision --index-url https://download.pytorch.org/whl/cpu
|
||||
RUN uv pip install transformers protobuf langchain sentence-transformers faiss-cpu nltk beautifulsoup4 bitsandbytes tabulate icetk --upgrade
|
||||
RUN uv pip install unstructured[all-docs] --upgrade
|
||||
ENV PATH="/gpt/.venv/bin:$PATH"
|
||||
RUN python -c 'import loguru'
|
||||
|
||||
# 装载项目文件,安装剩余依赖(必要)
|
||||
COPY . .
|
||||
RUN uv pip install -r requirements.txt -i https://mirrors.aliyun.com/pypi/simple/
|
||||
|
||||
# 安装依赖
|
||||
RUN pip3 install -r requirements.txt
|
||||
# # 非必要步骤,用于预热模块(可以删除)
|
||||
RUN python -c 'from check_proxy import warm_up_modules; warm_up_modules()'
|
||||
|
||||
# 安装知识库插件的额外依赖
|
||||
RUN apt-get update && apt-get install libgl1 -y
|
||||
RUN pip3 install torch torchvision --index-url https://download.pytorch.org/whl/cpu
|
||||
RUN pip3 install transformers protobuf langchain sentence-transformers faiss-cpu nltk beautifulsoup4 bitsandbytes tabulate icetk --upgrade
|
||||
RUN pip3 install unstructured[all-docs] --upgrade
|
||||
RUN python3 -c 'from check_proxy import warm_up_vectordb; warm_up_vectordb()'
|
||||
|
||||
# edge-tts需要的依赖
|
||||
RUN apt update && apt install ffmpeg -y
|
||||
|
||||
# 可选步骤,用于预热模块
|
||||
RUN python3 -c 'from check_proxy import warm_up_modules; warm_up_modules()'
|
||||
RUN python3 -m pip cache purge && apt-get clean
|
||||
ENV CGO_ENABLED=0
|
||||
|
||||
# 启动(必要)
|
||||
CMD ["bash", "-c", "python main.py"]
|
||||
|
||||
# 启动
|
||||
CMD ["python3", "-u", "main.py"]
|
||||
|
||||
@@ -19,24 +19,24 @@
|
||||
| crazy_functions\Latex全文润色.py | 对整个 Latex 项目进行润色和纠错 |
|
||||
| crazy_functions\Latex全文翻译.py | 对整个 Latex 项目进行翻译 |
|
||||
| crazy_functions\\_\_init\_\_.py | 模块初始化文件,标识 `crazy_functions` 是一个包 |
|
||||
| crazy_functions\下载arxiv论文翻译摘要.py | 下载 `arxiv` 论文的 PDF 文件,并提取摘要和翻译 |
|
||||
| crazy_functions\Arxiv_Downloader.py | 下载 `arxiv` 论文的 PDF 文件,并提取摘要和翻译 |
|
||||
| crazy_functions\代码重写为全英文_多线程.py | 将Python源代码文件中的中文内容转化为英文 |
|
||||
| crazy_functions\图片生成.py | 根据激励文本使用GPT模型生成相应的图像 |
|
||||
| crazy_functions\Conversation_To_File.py | 将每次对话记录写入Markdown格式的文件中 |
|
||||
| crazy_functions\总结word文档.py | 对输入的word文档进行摘要生成 |
|
||||
| crazy_functions\总结音视频.py | 对输入的音视频文件进行摘要生成 |
|
||||
| crazy_functions\Word_Summary.py | 对输入的word文档进行摘要生成 |
|
||||
| crazy_functions\Audio_Summary.py | 对输入的音视频文件进行摘要生成 |
|
||||
| crazy_functions\Markdown_Translate.py | 将指定目录下的Markdown文件进行中英文翻译 |
|
||||
| crazy_functions\批量总结PDF文档.py | 对PDF文件进行切割和摘要生成 |
|
||||
| crazy_functions\批量总结PDF文档pdfminer.py | 对PDF文件进行文本内容的提取和摘要生成 |
|
||||
| crazy_functions\PDF_Summary.py | 对PDF文件进行切割和摘要生成 |
|
||||
| crazy_functions\PDF_Summarypdfminer.py | 对PDF文件进行文本内容的提取和摘要生成 |
|
||||
| crazy_functions\PDF_Translate.py | 将指定目录下的PDF文件进行中英文翻译 |
|
||||
| crazy_functions\理解PDF文档内容.py | 对PDF文件进行摘要生成和问题解答 |
|
||||
| crazy_functions\生成函数注释.py | 自动生成Python函数的注释 |
|
||||
| crazy_functions\联网的ChatGPT.py | 使用网络爬虫和ChatGPT模型进行聊天回答 |
|
||||
| crazy_functions\解析JupyterNotebook.py | 对Jupyter Notebook进行代码解析 |
|
||||
| crazy_functions\PDF_QA.py | 对PDF文件进行摘要生成和问题解答 |
|
||||
| crazy_functions\Program_Comment_Gen.py | 自动生成Python函数的注释 |
|
||||
| crazy_functions\Internet_GPT_Legacy.py | 使用网络爬虫和ChatGPT模型进行聊天回答 |
|
||||
| crazy_functions\SourceCode_Analyse_JupyterNotebook.py | 对Jupyter Notebook进行代码解析 |
|
||||
| crazy_functions\解析项目源代码.py | 对指定编程语言的源代码进行解析 |
|
||||
| crazy_functions\询问多个大语言模型.py | 使用多个大语言模型对输入进行处理和回复 |
|
||||
| crazy_functions\读文章写摘要.py | 对论文进行解析和全文摘要生成 |
|
||||
| crazy_functions\谷歌检索小助手.py | 提供谷歌学术搜索页面中相关文章的元数据信息。 |
|
||||
| crazy_functions\Multi_LLM_Query.py | 使用多个大语言模型对输入进行处理和回复 |
|
||||
| crazy_functions\Paper_Abstract_Writer.py | 对论文进行解析和全文摘要生成 |
|
||||
| crazy_functions\Google_Scholar_Assistant_Legacy.py | 提供谷歌学术搜索页面中相关文章的元数据信息。 |
|
||||
| crazy_functions\高级功能函数模板.py | 使用Unsplash API发送相关图片以回复用户的输入。 |
|
||||
| request_llms\bridge_all.py | 基于不同LLM模型进行对话。 |
|
||||
| request_llms\bridge_chatglm.py | 使用ChatGLM模型生成回复,支持单线程和多线程方式。 |
|
||||
@@ -120,7 +120,7 @@ toolbox.py是一个工具类库,其中主要包含了一些函数装饰器和
|
||||
|
||||
## [10/48] 请对下面的程序文件做一个概述: crazy_functions\crazy_functions_test.py
|
||||
|
||||
这个文件是一个Python测试模块,用于测试crazy_functions中的各种函数插件。这些函数包括:解析Python项目源代码、解析Cpp项目源代码、Latex全文润色、Markdown中译英、批量翻译PDF文档、谷歌检索小助手、总结word文档、下载arxiv论文并翻译摘要、联网回答问题、和解析Jupyter Notebooks。对于每个函数插件,都有一个对应的测试函数来进行测试。
|
||||
这个文件是一个Python测试模块,用于测试crazy_functions中的各种函数插件。这些函数包括:解析Python项目源代码、解析Cpp项目源代码、Latex全文润色、Markdown中译英、批量翻译PDF文档、Google_Scholar_Assistant_Legacy、Word_Summary、下载arxiv论文并翻译摘要、联网回答问题、和解析Jupyter Notebooks。对于每个函数插件,都有一个对应的测试函数来进行测试。
|
||||
|
||||
## [11/48] 请对下面的程序文件做一个概述: crazy_functions\crazy_utils.py
|
||||
|
||||
@@ -143,9 +143,9 @@ toolbox.py是一个工具类库,其中主要包含了一些函数装饰器和
|
||||
|
||||
这是一个Python模块的初始化文件(__init__.py),命名为"crazy_functions"。该模块包含了一些疯狂的函数,但该文件并没有实现这些函数,而是作为一个包(package)来导入其它的Python模块以实现这些函数。在该文件中,没有定义任何类或函数,它唯一的作用就是标识"crazy_functions"模块是一个包。
|
||||
|
||||
## [15/48] 请对下面的程序文件做一个概述: crazy_functions\下载arxiv论文翻译摘要.py
|
||||
## [15/48] 请对下面的程序文件做一个概述: crazy_functions\Arxiv_Downloader.py
|
||||
|
||||
这是一个 Python 程序文件,文件名为 `下载arxiv论文翻译摘要.py`。程序包含多个函数,其中 `下载arxiv论文并翻译摘要` 函数的作用是下载 `arxiv` 论文的 PDF 文件,提取摘要并使用 GPT 对其进行翻译。其他函数包括用于下载 `arxiv` 论文的 `download_arxiv_` 函数和用于获取文章信息的 `get_name` 函数,其中涉及使用第三方库如 requests, BeautifulSoup 等。该文件还包含一些用于调试和存储文件的代码段。
|
||||
这是一个 Python 程序文件,文件名为 `Arxiv_Downloader.py`。程序包含多个函数,其中 `下载arxiv论文并翻译摘要` 函数的作用是下载 `arxiv` 论文的 PDF 文件,提取摘要并使用 GPT 对其进行翻译。其他函数包括用于下载 `arxiv` 论文的 `download_arxiv_` 函数和用于获取文章信息的 `get_name` 函数,其中涉及使用第三方库如 requests, BeautifulSoup 等。该文件还包含一些用于调试和存储文件的代码段。
|
||||
|
||||
## [16/48] 请对下面的程序文件做一个概述: crazy_functions\代码重写为全英文_多线程.py
|
||||
|
||||
@@ -167,11 +167,11 @@ toolbox.py是一个工具类库,其中主要包含了一些函数装饰器和
|
||||
|
||||
4. Conversation_To_File(txt, llm_kwargs, plugin_kwargs, chatbot, history, system_prompt, user_request):一个主要函数,用于保存当前对话记录并提醒用户。如果用户希望加载历史记录,则调用read_file_to_chat()来更新聊天显示框。如果用户希望删除历史记录,调用删除所有本地对话历史记录()函数完成删除操作。
|
||||
|
||||
## [19/48] 请对下面的程序文件做一个概述: crazy_functions\总结word文档.py
|
||||
## [19/48] 请对下面的程序文件做一个概述: crazy_functions\Word_Summary.py
|
||||
|
||||
该程序文件实现了一个总结Word文档的功能,使用Python的docx库读取docx格式的文件,使用pywin32库读取doc格式的文件。程序会先根据传入的txt参数搜索需要处理的文件,并逐个解析其中的内容,将内容拆分为指定长度的文章片段,然后使用另一个程序文件中的request_gpt_model_in_new_thread_with_ui_alive函数进行中文概述。最后将所有的总结结果写入一个文件中,并在界面上进行展示。
|
||||
该程序文件实现了一个Word_Summary的功能,使用Python的docx库读取docx格式的文件,使用pywin32库读取doc格式的文件。程序会先根据传入的txt参数搜索需要处理的文件,并逐个解析其中的内容,将内容拆分为指定长度的文章片段,然后使用另一个程序文件中的request_gpt_model_in_new_thread_with_ui_alive函数进行中文概述。最后将所有的总结结果写入一个文件中,并在界面上进行展示。
|
||||
|
||||
## [20/48] 请对下面的程序文件做一个概述: crazy_functions\总结音视频.py
|
||||
## [20/48] 请对下面的程序文件做一个概述: crazy_functions\Audio_Summary.py
|
||||
|
||||
该程序文件包括两个函数:split_audio_file()和AnalyAudio(),并且导入了一些必要的库并定义了一些工具函数。split_audio_file用于将音频文件分割成多个时长相等的片段,返回一个包含所有切割音频片段文件路径的列表,而AnalyAudio用来分析音频文件,通过调用whisper模型进行音频转文字并使用GPT模型对音频内容进行概述,最终将所有总结结果写入结果文件中。
|
||||
|
||||
@@ -179,31 +179,31 @@ toolbox.py是一个工具类库,其中主要包含了一些函数装饰器和
|
||||
|
||||
该程序文件名为`Markdown_Translate.py`,包含了以下功能:读取Markdown文件,将长文本分离开来,将Markdown文件进行翻译(英译中和中译英),整理结果并退出。程序使用了多线程以提高效率。程序使用了`tiktoken`依赖库,可能需要额外安装。文件中还有一些其他的函数和类,但与文件名所描述的功能无关。
|
||||
|
||||
## [22/48] 请对下面的程序文件做一个概述: crazy_functions\批量总结PDF文档.py
|
||||
## [22/48] 请对下面的程序文件做一个概述: crazy_functions\PDF_Summary.py
|
||||
|
||||
该文件是一个Python脚本,名为crazy_functions\批量总结PDF文档.py。在导入了一系列库和工具函数后,主要定义了5个函数,其中包括一个错误处理装饰器(@CatchException),用于批量总结PDF文档。该函数主要实现对PDF文档的解析,并调用模型生成中英文摘要。
|
||||
该文件是一个Python脚本,名为crazy_functions\PDF_Summary.py。在导入了一系列库和工具函数后,主要定义了5个函数,其中包括一个错误处理装饰器(@CatchException),用于PDF_Summary。该函数主要实现对PDF文档的解析,并调用模型生成中英文摘要。
|
||||
|
||||
## [23/48] 请对下面的程序文件做一个概述: crazy_functions\批量总结PDF文档pdfminer.py
|
||||
## [23/48] 请对下面的程序文件做一个概述: crazy_functions\PDF_Summarypdfminer.py
|
||||
|
||||
该程序文件是一个用于批量总结PDF文档的函数插件,使用了pdfminer插件和BeautifulSoup库来提取PDF文档的文本内容,对每个PDF文件分别进行处理并生成中英文摘要。同时,该程序文件还包括一些辅助工具函数和处理异常的装饰器。
|
||||
该程序文件是一个用于PDF_Summary的函数插件,使用了pdfminer插件和BeautifulSoup库来提取PDF文档的文本内容,对每个PDF文件分别进行处理并生成中英文摘要。同时,该程序文件还包括一些辅助工具函数和处理异常的装饰器。
|
||||
|
||||
## [24/48] 请对下面的程序文件做一个概述: crazy_functions\PDF_Translate.py
|
||||
|
||||
这个程序文件是一个Python脚本,文件名为“PDF_Translate.py”。它主要使用了“toolbox”、“request_gpt_model_in_new_thread_with_ui_alive”、“request_gpt_model_multi_threads_with_very_awesome_ui_and_high_efficiency”、“colorful”等Python库和自定义的模块“crazy_utils”的一些函数。程序实现了一个批量翻译PDF文档的功能,可以自动解析PDF文件中的基础信息,递归地切割PDF文件,翻译和处理PDF论文中的所有内容,并生成相应的翻译结果文件(包括md文件和html文件)。功能比较复杂,其中需要调用多个函数和依赖库,涉及到多线程操作和UI更新。文件中有详细的注释和变量命名,代码比较清晰易读。
|
||||
|
||||
## [25/48] 请对下面的程序文件做一个概述: crazy_functions\理解PDF文档内容.py
|
||||
## [25/48] 请对下面的程序文件做一个概述: crazy_functions\PDF_QA.py
|
||||
|
||||
该程序文件实现了一个名为“理解PDF文档内容”的函数,该函数可以为输入的PDF文件提取摘要以及正文各部分的主要内容,并在提取过程中根据上下文关系进行学术性问题解答。该函数依赖于多个辅助函数和第三方库,并在执行过程中针对可能出现的异常进行了处理。
|
||||
该程序文件实现了一个名为“PDF_QA”的函数,该函数可以为输入的PDF文件提取摘要以及正文各部分的主要内容,并在提取过程中根据上下文关系进行学术性问题解答。该函数依赖于多个辅助函数和第三方库,并在执行过程中针对可能出现的异常进行了处理。
|
||||
|
||||
## [26/48] 请对下面的程序文件做一个概述: crazy_functions\生成函数注释.py
|
||||
## [26/48] 请对下面的程序文件做一个概述: crazy_functions\Program_Comment_Gen.py
|
||||
|
||||
该程序文件是一个Python模块文件,文件名为“生成函数注释.py”,定义了两个函数:一个是生成函数注释的主函数“生成函数注释”,另一个是通过装饰器实现异常捕捉的函数“批量生成函数注释”。该程序文件依赖于“toolbox”和本地“crazy_utils”模块,并且在运行时使用了多线程技术和GPT模型来生成注释。函数生成的注释结果使用Markdown表格输出并写入历史记录文件。
|
||||
该程序文件是一个Python模块文件,文件名为“Program_Comment_Gen.py”,定义了两个函数:一个是Program_Comment_Gen的主函数“Program_Comment_Gen”,另一个是通过装饰器实现异常捕捉的函数“批量Program_Comment_Gen”。该程序文件依赖于“toolbox”和本地“crazy_utils”模块,并且在运行时使用了多线程技术和GPT模型来生成注释。函数生成的注释结果使用Markdown表格输出并写入历史记录文件。
|
||||
|
||||
## [27/48] 请对下面的程序文件做一个概述: crazy_functions\联网的ChatGPT.py
|
||||
## [27/48] 请对下面的程序文件做一个概述: crazy_functions\Internet_GPT_Legacy.py
|
||||
|
||||
这是一个名为`联网的ChatGPT.py`的Python程序文件,其中定义了一个函数`连接网络回答问题`。该函数通过爬取搜索引擎的结果和访问网页来综合回答给定的问题,并使用ChatGPT模型完成回答。此外,该文件还包括一些工具函数,例如从网页中抓取文本和使用代理访问网页。
|
||||
这是一个名为`Internet_GPT_Legacy.py`的Python程序文件,其中定义了一个函数`连接网络回答问题`。该函数通过爬取搜索引擎的结果和访问网页来综合回答给定的问题,并使用ChatGPT模型完成回答。此外,该文件还包括一些工具函数,例如从网页中抓取文本和使用代理访问网页。
|
||||
|
||||
## [28/48] 请对下面的程序文件做一个概述: crazy_functions\解析JupyterNotebook.py
|
||||
## [28/48] 请对下面的程序文件做一个概述: crazy_functions\SourceCode_Analyse_JupyterNotebook.py
|
||||
|
||||
这个程序文件包含了两个函数: `parseNotebook()`和`解析ipynb文件()`,并且引入了一些工具函数和类。`parseNotebook()`函数将Jupyter Notebook文件解析为文本代码块,`解析ipynb文件()`函数则用于解析多个Jupyter Notebook文件,使用`parseNotebook()`解析每个文件和一些其他的处理。函数中使用了多线程处理输入和输出,并且将结果写入到文件中。
|
||||
|
||||
@@ -211,17 +211,17 @@ toolbox.py是一个工具类库,其中主要包含了一些函数装饰器和
|
||||
|
||||
这是一个源代码分析的Python代码文件,其中定义了多个函数,包括解析一个Python项目、解析一个C项目、解析一个C项目的头文件和解析一个Java项目等。其中解析源代码新函数是实际处理源代码分析并生成报告的函数。该函数首先会逐个读取传入的源代码文件,生成对应的请求内容,通过多线程发送到chatgpt进行分析。然后将结果写入文件,并进行汇总分析。最后通过调用update_ui函数刷新界面,完整实现了源代码的分析。
|
||||
|
||||
## [30/48] 请对下面的程序文件做一个概述: crazy_functions\询问多个大语言模型.py
|
||||
## [30/48] 请对下面的程序文件做一个概述: crazy_functions\Multi_LLM_Query.py
|
||||
|
||||
该程序文件包含两个函数:同时问询()和同时问询_指定模型(),它们的作用是使用多个大语言模型同时对用户输入进行处理,返回对应模型的回复结果。同时问询()会默认使用ChatGPT和ChatGLM两个模型,而同时问询_指定模型()则可以指定要使用的模型。该程序文件还引用了其他的模块和函数库。
|
||||
|
||||
## [31/48] 请对下面的程序文件做一个概述: crazy_functions\读文章写摘要.py
|
||||
## [31/48] 请对下面的程序文件做一个概述: crazy_functions\Paper_Abstract_Writer.py
|
||||
|
||||
这个程序文件是一个Python模块,文件名为crazy_functions\读文章写摘要.py。该模块包含了两个函数,其中主要函数是"读文章写摘要"函数,其实现了解析给定文件夹中的tex文件,对其中每个文件的内容进行摘要生成,并根据各论文片段的摘要,最终生成全文摘要。第二个函数是"解析Paper"函数,用于解析单篇论文文件。其中用到了一些工具函数和库,如update_ui、CatchException、report_exception、write_results_to_file等。
|
||||
这个程序文件是一个Python模块,文件名为crazy_functions\Paper_Abstract_Writer.py。该模块包含了两个函数,其中主要函数是"Paper_Abstract_Writer"函数,其实现了解析给定文件夹中的tex文件,对其中每个文件的内容进行摘要生成,并根据各论文片段的摘要,最终生成全文摘要。第二个函数是"解析Paper"函数,用于解析单篇论文文件。其中用到了一些工具函数和库,如update_ui、CatchException、report_exception、write_results_to_file等。
|
||||
|
||||
## [32/48] 请对下面的程序文件做一个概述: crazy_functions\谷歌检索小助手.py
|
||||
## [32/48] 请对下面的程序文件做一个概述: crazy_functions\Google_Scholar_Assistant_Legacy.py
|
||||
|
||||
该文件是一个Python模块,文件名为“谷歌检索小助手.py”。该模块包含两个函数,一个是“get_meta_information()”,用于从提供的网址中分析出所有相关的学术文献的元数据信息;另一个是“谷歌检索小助手()”,是主函数,用于分析用户提供的谷歌学术搜索页面中出现的文章,并提取相关信息。其中,“谷歌检索小助手()”函数依赖于“get_meta_information()”函数,并调用了其他一些Python模块,如“arxiv”、“math”、“bs4”等。
|
||||
该文件是一个Python模块,文件名为“Google_Scholar_Assistant_Legacy.py”。该模块包含两个函数,一个是“get_meta_information()”,用于从提供的网址中分析出所有相关的学术文献的元数据信息;另一个是“Google_Scholar_Assistant_Legacy()”,是主函数,用于分析用户提供的谷歌学术搜索页面中出现的文章,并提取相关信息。其中,“Google_Scholar_Assistant_Legacy()”函数依赖于“get_meta_information()”函数,并调用了其他一些Python模块,如“arxiv”、“math”、“bs4”等。
|
||||
|
||||
## [33/48] 请对下面的程序文件做一个概述: crazy_functions\高级功能函数模板.py
|
||||
|
||||
@@ -307,7 +307,7 @@ GetGLMHandle 类继承自Process类(多进程),主要功能是启动一个
|
||||
这个文件是用于对llm模型进行单元测试的Python程序。程序导入一个名为"request_llms.bridge_newbingfree"的模块,然后三次使用该模块中的predict_no_ui_long_connection()函数进行预测,并输出结果。此外,还有一些注释掉的代码段,这些代码段也是关于模型预测的。
|
||||
|
||||
## 用一张Markdown表格简要描述以下文件的功能:
|
||||
check_proxy.py, colorful.py, config.py, config_private.py, core_functional.py, crazy_functional.py, main.py, multi_language.py, theme.py, toolbox.py, crazy_functions\crazy_functions_test.py, crazy_functions\crazy_utils.py, crazy_functions\Latex全文润色.py, crazy_functions\Latex全文翻译.py, crazy_functions\__init__.py, crazy_functions\下载arxiv论文翻译摘要.py。根据以上分析,用一句话概括程序的整体功能。
|
||||
check_proxy.py, colorful.py, config.py, config_private.py, core_functional.py, crazy_functional.py, main.py, multi_language.py, theme.py, toolbox.py, crazy_functions\crazy_functions_test.py, crazy_functions\crazy_utils.py, crazy_functions\Latex全文润色.py, crazy_functions\Latex全文翻译.py, crazy_functions\__init__.py, crazy_functions\Arxiv_Downloader.py。根据以上分析,用一句话概括程序的整体功能。
|
||||
|
||||
| 文件名 | 功能描述 |
|
||||
| ------ | ------ |
|
||||
@@ -326,40 +326,40 @@ check_proxy.py, colorful.py, config.py, config_private.py, core_functional.py, c
|
||||
| crazy_functions\Latex全文润色.py | 对整个 Latex 项目进行润色和纠错 |
|
||||
| crazy_functions\Latex全文翻译.py | 对整个 Latex 项目进行翻译 |
|
||||
| crazy_functions\__init__.py | 模块初始化文件,标识 `crazy_functions` 是一个包 |
|
||||
| crazy_functions\下载arxiv论文翻译摘要.py | 下载 `arxiv` 论文的 PDF 文件,并提取摘要和翻译 |
|
||||
| crazy_functions\Arxiv_Downloader.py | 下载 `arxiv` 论文的 PDF 文件,并提取摘要和翻译 |
|
||||
|
||||
这些程序源文件提供了基础的文本和语言处理功能、工具函数和高级插件,使 Chatbot 能够处理各种复杂的学术文本问题,包括润色、翻译、搜索、下载、解析等。
|
||||
|
||||
## 用一张Markdown表格简要描述以下文件的功能:
|
||||
crazy_functions\代码重写为全英文_多线程.py, crazy_functions\图片生成.py, crazy_functions\Conversation_To_File.py, crazy_functions\总结word文档.py, crazy_functions\总结音视频.py, crazy_functions\Markdown_Translate.py, crazy_functions\批量总结PDF文档.py, crazy_functions\批量总结PDF文档pdfminer.py, crazy_functions\PDF_Translate.py, crazy_functions\理解PDF文档内容.py, crazy_functions\生成函数注释.py, crazy_functions\联网的ChatGPT.py, crazy_functions\解析JupyterNotebook.py, crazy_functions\解析项目源代码.py, crazy_functions\询问多个大语言模型.py, crazy_functions\读文章写摘要.py。根据以上分析,用一句话概括程序的整体功能。
|
||||
crazy_functions\代码重写为全英文_多线程.py, crazy_functions\图片生成.py, crazy_functions\Conversation_To_File.py, crazy_functions\Word_Summary.py, crazy_functions\Audio_Summary.py, crazy_functions\Markdown_Translate.py, crazy_functions\PDF_Summary.py, crazy_functions\PDF_Summarypdfminer.py, crazy_functions\PDF_Translate.py, crazy_functions\PDF_QA.py, crazy_functions\Program_Comment_Gen.py, crazy_functions\Internet_GPT_Legacy.py, crazy_functions\SourceCode_Analyse_JupyterNotebook.py, crazy_functions\解析项目源代码.py, crazy_functions\Multi_LLM_Query.py, crazy_functions\Paper_Abstract_Writer.py。根据以上分析,用一句话概括程序的整体功能。
|
||||
|
||||
| 文件名 | 功能简述 |
|
||||
| --- | --- |
|
||||
| 代码重写为全英文_多线程.py | 将Python源代码文件中的中文内容转化为英文 |
|
||||
| 图片生成.py | 根据激励文本使用GPT模型生成相应的图像 |
|
||||
| Conversation_To_File.py | 将每次对话记录写入Markdown格式的文件中 |
|
||||
| 总结word文档.py | 对输入的word文档进行摘要生成 |
|
||||
| 总结音视频.py | 对输入的音视频文件进行摘要生成 |
|
||||
| Word_Summary.py | 对输入的word文档进行摘要生成 |
|
||||
| Audio_Summary.py | 对输入的音视频文件进行摘要生成 |
|
||||
| Markdown_Translate.py | 将指定目录下的Markdown文件进行中英文翻译 |
|
||||
| 批量总结PDF文档.py | 对PDF文件进行切割和摘要生成 |
|
||||
| 批量总结PDF文档pdfminer.py | 对PDF文件进行文本内容的提取和摘要生成 |
|
||||
| PDF_Summary.py | 对PDF文件进行切割和摘要生成 |
|
||||
| PDF_Summarypdfminer.py | 对PDF文件进行文本内容的提取和摘要生成 |
|
||||
| PDF_Translate.py | 将指定目录下的PDF文件进行中英文翻译 |
|
||||
| 理解PDF文档内容.py | 对PDF文件进行摘要生成和问题解答 |
|
||||
| 生成函数注释.py | 自动生成Python函数的注释 |
|
||||
| 联网的ChatGPT.py | 使用网络爬虫和ChatGPT模型进行聊天回答 |
|
||||
| 解析JupyterNotebook.py | 对Jupyter Notebook进行代码解析 |
|
||||
| PDF_QA.py | 对PDF文件进行摘要生成和问题解答 |
|
||||
| Program_Comment_Gen.py | 自动生成Python函数的注释 |
|
||||
| Internet_GPT_Legacy.py | 使用网络爬虫和ChatGPT模型进行聊天回答 |
|
||||
| SourceCode_Analyse_JupyterNotebook.py | 对Jupyter Notebook进行代码解析 |
|
||||
| 解析项目源代码.py | 对指定编程语言的源代码进行解析 |
|
||||
| 询问多个大语言模型.py | 使用多个大语言模型对输入进行处理和回复 |
|
||||
| 读文章写摘要.py | 对论文进行解析和全文摘要生成 |
|
||||
| Multi_LLM_Query.py | 使用多个大语言模型对输入进行处理和回复 |
|
||||
| Paper_Abstract_Writer.py | 对论文进行解析和全文摘要生成 |
|
||||
|
||||
概括程序的整体功能:提供了一系列处理文本、文件和代码的功能,使用了各类语言模型、多线程、网络请求和数据解析技术来提高效率和精度。
|
||||
|
||||
## 用一张Markdown表格简要描述以下文件的功能:
|
||||
crazy_functions\谷歌检索小助手.py, crazy_functions\高级功能函数模板.py, request_llms\bridge_all.py, request_llms\bridge_chatglm.py, request_llms\bridge_chatgpt.py, request_llms\bridge_jittorllms_llama.py, request_llms\bridge_jittorllms_pangualpha.py, request_llms\bridge_jittorllms_rwkv.py, request_llms\bridge_moss.py, request_llms\bridge_newbing.py, request_llms\bridge_newbingfree.py, request_llms\bridge_stackclaude.py, request_llms\bridge_tgui.py, request_llms\edge_gpt.py, request_llms\edge_gpt_free.py, request_llms\test_llms.py。根据以上分析,用一句话概括程序的整体功能。
|
||||
crazy_functions\Google_Scholar_Assistant_Legacy.py, crazy_functions\高级功能函数模板.py, request_llms\bridge_all.py, request_llms\bridge_chatglm.py, request_llms\bridge_chatgpt.py, request_llms\bridge_jittorllms_llama.py, request_llms\bridge_jittorllms_pangualpha.py, request_llms\bridge_jittorllms_rwkv.py, request_llms\bridge_moss.py, request_llms\bridge_newbing.py, request_llms\bridge_newbingfree.py, request_llms\bridge_stackclaude.py, request_llms\bridge_tgui.py, request_llms\edge_gpt.py, request_llms\edge_gpt_free.py, request_llms\test_llms.py。根据以上分析,用一句话概括程序的整体功能。
|
||||
|
||||
| 文件名 | 功能描述 |
|
||||
| --- | --- |
|
||||
| crazy_functions\谷歌检索小助手.py | 提供谷歌学术搜索页面中相关文章的元数据信息。 |
|
||||
| crazy_functions\Google_Scholar_Assistant_Legacy.py | 提供谷歌学术搜索页面中相关文章的元数据信息。 |
|
||||
| crazy_functions\高级功能函数模板.py | 使用Unsplash API发送相关图片以回复用户的输入。 |
|
||||
| request_llms\bridge_all.py | 基于不同LLM模型进行对话。 |
|
||||
| request_llms\bridge_chatglm.py | 使用ChatGLM模型生成回复,支持单线程和多线程方式。 |
|
||||
|
||||
@@ -347,7 +347,7 @@ def markdown_convertion(txt):
|
||||
|
||||
# 在文本中插入一个base64编码的原始文本,以便在复制时能够获得原始文本
|
||||
raw_text_encoded = compress_string(txt)
|
||||
raw_text_node = f'<div class="raw_text" style="display:none">{raw_text_encoded}</div><div class="message_tail" style="display:none"/>'
|
||||
raw_text_node = f'<div class="raw_text" style="display:none">{raw_text_encoded}</div><div class="message_tail" style="display:none"></div>'
|
||||
suf = raw_text_node + "</div>"
|
||||
|
||||
# 用于查找数学公式的正则表达式
|
||||
@@ -407,7 +407,7 @@ def code_block_title_replace_format(match):
|
||||
|
||||
|
||||
def get_last_backticks_indent(text):
|
||||
# 从后向前查找最后一个 ```
|
||||
# 从后向前查找最后一个 ```
|
||||
lines = text.splitlines()
|
||||
for line in reversed(lines):
|
||||
if '```' in line:
|
||||
|
||||
@@ -0,0 +1,622 @@
|
||||
"""
|
||||
FastAPI WebSocket 流式服务器模块
|
||||
|
||||
这个模块实现了一个基于FastAPI的WebSocket服务器,用于处理前端和后端之间的实时双向通信。
|
||||
主要功能包括:
|
||||
1. 维护WebSocket连接
|
||||
2. 处理消息队列
|
||||
3. 支持实时和阻塞式消息传输
|
||||
4. 提供事件管理机制
|
||||
|
||||
主要组件:
|
||||
- FutureEvent: 用于事件管理的线程事件类
|
||||
- UserInterfaceMsg: 定义前后端通信的消息格式
|
||||
- PythonMethod_AsyncConnectionMaintainer_AgentcraftInterface: 连接维护器
|
||||
- MasterMindWebSocketServer: WebSocket服务器主类
|
||||
"""
|
||||
|
||||
import uuid
|
||||
import time
|
||||
import json
|
||||
import platform
|
||||
import pickle
|
||||
import asyncio
|
||||
import threading
|
||||
import traceback
|
||||
from fastapi.websockets import WebSocketState
|
||||
from loguru import logger
|
||||
from queue import Queue
|
||||
from fastapi import FastAPI, WebSocket, File, UploadFile, Form
|
||||
from fastapi.responses import JSONResponse
|
||||
from datetime import datetime
|
||||
from typing import List, Optional, Any, Union
|
||||
from pydantic import BaseModel, Field
|
||||
import shutil
|
||||
import os
|
||||
from enum import auto, Enum
|
||||
from typing import List
|
||||
from toolbox import ChatBotWithCookies
|
||||
import starlette
|
||||
|
||||
|
||||
|
||||
class UserInterfaceMsg(BaseModel):
|
||||
"""
|
||||
用户界面消息模型,定义了前后端通信的数据结构
|
||||
|
||||
这个类使用Pydantic BaseModel定义了所有可能的消息字段,
|
||||
包括插件功能、输入内容、LLM参数、聊天记录等。
|
||||
"""
|
||||
function: str = Field(default="chat", description="使用哪个插件")
|
||||
main_input: str = Field(default="", description="主要输入内容,通常是用户的问题")
|
||||
llm_kwargs: dict = Field(default_factory=dict, description="围绕LLM的各种参数")
|
||||
# llm_kwargs = {
|
||||
# "api_key": cookies["api_key"],
|
||||
# "llm_model": cookies["llm_model"],
|
||||
# "top_p": 1.0,
|
||||
# "max_length": None,
|
||||
# "temperature": 1.0,
|
||||
# "user_name": "default_user", # Default user name
|
||||
# }
|
||||
plugin_kwargs: dict = Field(default_factory=dict, description="围绕该function的各种参数")
|
||||
chatbot: list[list[Union[str|None]]] = Field(default=[], description="聊天记录(给人类看的)。格式为 [ [user_msg, bot_msg], [user_msg_2, bot_msg_2],...],双层列表,第一层是每一轮对话,第二层是用户和机器人的消息。")
|
||||
chatbot_cookies: dict = Field(default_factory=dict, description="其他新前端涉及的参数")
|
||||
history: list[str] = Field(default=[], description="聊天记录(给模型看的)。单层列表")
|
||||
system_prompt: str = Field(default="", description="系统提示词")
|
||||
user_request: dict = Field(default="", description="用户相关的参数,如用户名")
|
||||
special_kwargs: dict = Field(default_factory=dict, description="其他新前端涉及的参数")
|
||||
special_state: dict = Field(default={}, description="特殊状态传递,例如对话结束。")
|
||||
|
||||
TERMINATE_MSG = UserInterfaceMsg(function="TERMINATE", special_state={"stop": True})
|
||||
|
||||
def setup_initial_com(initial_msg: UserInterfaceMsg):
|
||||
"""
|
||||
设置插件参数
|
||||
|
||||
从初始消息中提取各种参数并构建插件执行所需的参数字典。
|
||||
|
||||
参数:
|
||||
initial_msg: 初始的用户消息
|
||||
chatbot_with_cookies: 带有cookie的聊天机器人实例
|
||||
|
||||
返回:
|
||||
dict: 包含插件执行所需所有参数的字典
|
||||
"""
|
||||
from toolbox import get_plugin_default_kwargs
|
||||
|
||||
|
||||
com = get_plugin_default_kwargs()
|
||||
com["main_input"] = initial_msg.main_input
|
||||
# 设置LLM相关参数
|
||||
if initial_msg.llm_kwargs.get('api_key', None): com["llm_kwargs"]['api_key'] = initial_msg.llm_kwargs.get('api_key')
|
||||
if initial_msg.llm_kwargs.get('llm_model', None): com["llm_kwargs"]['llm_model'] = initial_msg.llm_kwargs.get('llm_model')
|
||||
if initial_msg.llm_kwargs.get('top_p', None): com["llm_kwargs"]['top_p'] = initial_msg.llm_kwargs.get('top_p')
|
||||
if initial_msg.llm_kwargs.get('max_length', None): com["llm_kwargs"]['max_length'] = initial_msg.llm_kwargs.get('max_length')
|
||||
if initial_msg.llm_kwargs.get('temperature', None): com["llm_kwargs"]['temperature'] = initial_msg.llm_kwargs.get('temperature')
|
||||
if initial_msg.llm_kwargs.get('user_name', None): com["llm_kwargs"]['user_name'] = initial_msg.llm_kwargs.get('user_name')
|
||||
if initial_msg.llm_kwargs.get('embed_model', None): com["llm_kwargs"]['embed_model'] = initial_msg.llm_kwargs.get('embed_model')
|
||||
|
||||
initial_msg.chatbot_cookies.update({
|
||||
'api_key': com["llm_kwargs"]['api_key'],
|
||||
'top_p': com["llm_kwargs"]['top_p'],
|
||||
'llm_model': com["llm_kwargs"]['llm_model'],
|
||||
'embed_model': com["llm_kwargs"]['embed_model'],
|
||||
'temperature': com["llm_kwargs"]['temperature'],
|
||||
'user_name': com["llm_kwargs"]['user_name'],
|
||||
'customize_fn_overwrite': {},
|
||||
})
|
||||
chatbot_with_cookies = ChatBotWithCookies(initial_msg.chatbot_cookies)
|
||||
chatbot_with_cookies.write_list(initial_msg.chatbot)
|
||||
# 设置其他参数
|
||||
com["plugin_kwargs"] = initial_msg.plugin_kwargs
|
||||
com["chatbot_with_cookie"] = chatbot_with_cookies
|
||||
com["history"] = initial_msg.history
|
||||
com["system_prompt"] = initial_msg.system_prompt
|
||||
com["user_request"] = initial_msg.user_request
|
||||
|
||||
return com
|
||||
|
||||
|
||||
class DummyRequest(object):
|
||||
def __call__(self, username):
|
||||
self.username = username
|
||||
|
||||
|
||||
def task_executor(initial_msg:UserInterfaceMsg, queue_blocking_from_client:asyncio.Queue, queue_back_to_client:asyncio.Queue):
|
||||
"""
|
||||
initial_msg: 初始的用户消息 ( <---- begin_contact_websocket_server:initial_message )
|
||||
queue_blocking_from_client: 从客户端接收阻塞消息的队列
|
||||
queue_back_to_client: 发送消息回客户端的队列
|
||||
"""
|
||||
from toolbox import get_plugin_handle
|
||||
from toolbox import get_plugin_default_kwargs
|
||||
from toolbox import on_file_uploaded
|
||||
|
||||
def update_ui_websocket(chatbot: List[List[str]], history: List[str], chatbot_cookies: dict, special_state: dict):
|
||||
send_obj = UserInterfaceMsg(chatbot=chatbot, history=history, chatbot_cookies=chatbot_cookies, special_state=special_state)
|
||||
queue_back_to_client.put_nowait(send_obj)
|
||||
|
||||
|
||||
com = setup_initial_com(initial_msg)
|
||||
|
||||
main_input = com["main_input"]
|
||||
plugin_kwargs = com["plugin_kwargs"]
|
||||
chatbot_with_cookies = com["chatbot_with_cookie"]
|
||||
history = com["history"]
|
||||
system_prompt = com["system_prompt"]
|
||||
user_request = com["user_request"]
|
||||
llm_kwargs = com["llm_kwargs"]
|
||||
|
||||
|
||||
# ''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''
|
||||
# '''''''''''''''''''''''''''''''' 调用插件 '''''''''''''''''''''''''''''''''
|
||||
# ''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''
|
||||
# ''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''
|
||||
if initial_msg.function == "chat":
|
||||
plugin = get_plugin_handle("crazy_functions.Chat->Chat")
|
||||
my_working_plugin = (plugin)(**com)
|
||||
for cookies, chatbot, hist_json, msg in my_working_plugin:
|
||||
history = json.loads(hist_json)
|
||||
special_state = {'msg': msg}
|
||||
update_ui_websocket(chatbot=chatbot, history=history, chatbot_cookies=cookies, special_state=special_state) # ----> receive_callback_fn
|
||||
|
||||
if initial_msg.function == "basic":
|
||||
from request_llms.bridge_all import predict
|
||||
additional_fn = initial_msg.special_kwargs.get("core_function")
|
||||
for cookies, chatbot, hist_json, msg in \
|
||||
predict(main_input, llm_kwargs, plugin_kwargs, chatbot_with_cookies, history=history, system_prompt=system_prompt, stream = True, additional_fn=additional_fn):
|
||||
history = json.loads(hist_json)
|
||||
special_state = {'msg': msg}
|
||||
update_ui_websocket(chatbot=chatbot, history=history, chatbot_cookies=cookies, special_state=special_state) # ----> receive_callback_fn
|
||||
|
||||
# ''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''
|
||||
# '''''''''''''''''''''''''''''''' 上传文件 '''''''''''''''''''''''''''''''''
|
||||
# ''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''
|
||||
# ''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''
|
||||
if initial_msg.function == "upload":
|
||||
def wait_upload_done():
|
||||
print('Waiting for file upload to complete...')
|
||||
file_upload_done_msg = queue_blocking_from_client.get()
|
||||
print('file upload complete...')
|
||||
if file_upload_done_msg.function != "upload_done":
|
||||
raise ValueError("Expected 'upload_done' function, got: {}".format(file_upload_done_msg.function))
|
||||
if 'files' not in file_upload_done_msg.special_kwargs:
|
||||
raise ValueError("Expected 'files' in special_kwargs, got: {}".format(file_upload_done_msg.special_kwargs))
|
||||
return file_upload_done_msg.special_kwargs['files']
|
||||
|
||||
request = DummyRequest()
|
||||
request.username = "default_user"
|
||||
chatbot = initial_msg.chatbot
|
||||
history = initial_msg.history
|
||||
chatbot_cookies = initial_msg.chatbot_cookies
|
||||
special_state = {'msg': '等待上传'}
|
||||
chatbot += [[initial_msg.main_input, None]]
|
||||
update_ui_websocket(chatbot=chatbot, history=history, chatbot_cookies=chatbot_cookies, special_state=special_state) # ----> receive_callback_fn
|
||||
|
||||
# 等待上传完成
|
||||
files = wait_upload_done()
|
||||
|
||||
# 读取文件并预览
|
||||
chatbot += [['正在处理', None]]
|
||||
update_ui_websocket(chatbot=chatbot, history=history, chatbot_cookies=chatbot_cookies, special_state=special_state) # ----> receive_callback_fn
|
||||
chatbot, _, _, chatbot_cookies = on_file_uploaded(
|
||||
request=request, files=files, chatbot=chatbot,
|
||||
txt="", txt2="", checkboxes="", cookies=chatbot_cookies
|
||||
)
|
||||
special_state = {'msg': '完成上传'}
|
||||
# 更新前端
|
||||
update_ui_websocket(chatbot=chatbot, history=history, chatbot_cookies=chatbot_cookies, special_state=special_state) # ----> receive_callback_fn
|
||||
|
||||
|
||||
|
||||
class FutureEvent(threading.Event):
|
||||
"""
|
||||
扩展的线程事件类,用于异步操作的结果获取
|
||||
|
||||
这个类扩展了threading.Event,添加了返回值存储功能,
|
||||
使得可以在事件完成时同时传递结果数据。
|
||||
"""
|
||||
def __init__(self) -> None:
|
||||
super().__init__()
|
||||
self.return_value = None
|
||||
|
||||
def terminate(self, return_value):
|
||||
"""
|
||||
终止事件并设置返回值
|
||||
|
||||
参数:
|
||||
return_value: 事件完成时要返回的值
|
||||
"""
|
||||
self.return_value = return_value
|
||||
self.set()
|
||||
|
||||
def wait_and_get_result(self):
|
||||
"""
|
||||
等待事件完成并获取结果
|
||||
|
||||
返回:
|
||||
任意类型: 事件完成时设置的返回值
|
||||
"""
|
||||
self.wait()
|
||||
return self.return_value
|
||||
|
||||
|
||||
class AtomicQueue:
|
||||
|
||||
def __init__(self):
|
||||
self.queue = Queue()
|
||||
|
||||
def put(self, item):
|
||||
self.queue.put(item)
|
||||
|
||||
def put_nowait(self, item):
|
||||
self.queue.put(item)
|
||||
|
||||
def get(self, timeout=600):
|
||||
while self.queue.empty() and timeout > 0:
|
||||
time.sleep(1)
|
||||
timeout -= 1
|
||||
if timeout <= 0:
|
||||
raise TimeoutError("Queue get operation timed out")
|
||||
return self.queue.get()
|
||||
|
||||
|
||||
class PythonMethod_AsyncConnectionMaintainer_AgentcraftInterface():
|
||||
"""
|
||||
异步连接维护器接口类
|
||||
|
||||
负责维护WebSocket连接的核心类,处理消息队列的创建和管理,
|
||||
以及维护与客户端的长连接通信。
|
||||
"""
|
||||
|
||||
def make_queue(self):
|
||||
"""
|
||||
创建消息队列
|
||||
|
||||
创建三个异步队列用于不同类型的消息传输:
|
||||
1. queue_back_to_client: 发送给客户端的消息队列
|
||||
2. queue_realtime_from_client: 实时接收的客户端消息队列
|
||||
3. queue_blocking_from_client: 阻塞式接收的客户端消息队列
|
||||
|
||||
返回:
|
||||
tuple: 包含三个异步队列的元组
|
||||
"""
|
||||
queue_back_to_client = asyncio.Queue()
|
||||
queue_realtime_from_client = asyncio.Queue()
|
||||
queue_blocking_from_client = AtomicQueue()
|
||||
terminate_event = asyncio.Event()
|
||||
return queue_back_to_client, queue_realtime_from_client, queue_blocking_from_client, terminate_event
|
||||
|
||||
async def maintain_connection_forever(self, initial_msg: UserInterfaceMsg, websocket: WebSocket, client_id: str):
|
||||
"""
|
||||
永久维护WebSocket连接
|
||||
|
||||
处理与客户端的持续连接,包括消息的发送和接收。
|
||||
创建独立的任务处理消息发送和接收,并启动聊天处理线程。
|
||||
|
||||
参数:
|
||||
initial_msg: 初始消息
|
||||
websocket: WebSocket连接对象
|
||||
client_id: 客户端标识符
|
||||
"""
|
||||
|
||||
async def wait_message_to_send(queue_back_to_client: asyncio.Queue, terminate_event: asyncio.Event):
|
||||
"""
|
||||
等待并发送消息到客户端
|
||||
|
||||
持续监听消息队列,将消息序列化后发送给客户端。
|
||||
|
||||
参数:
|
||||
queue_back_to_client: 发送给客户端的消息队列
|
||||
"""
|
||||
# 🕜 wait message to send away -> front end
|
||||
msg_cnt = 0
|
||||
try:
|
||||
while True:
|
||||
|
||||
################
|
||||
# get message and check terminate
|
||||
while True:
|
||||
try:
|
||||
if terminate_event.is_set():
|
||||
msg = TERMINATE_MSG
|
||||
break
|
||||
else:
|
||||
msg: UserInterfaceMsg = await asyncio.wait_for(queue_back_to_client.get(), timeout=0.25)
|
||||
break
|
||||
except asyncio.TimeoutError:
|
||||
continue # 继续检查条件
|
||||
if msg.function == TERMINATE_MSG.function:
|
||||
logger.info("Received terminate message, skip this message and stopping wait_message_to_send.")
|
||||
break
|
||||
################
|
||||
|
||||
|
||||
msg_cnt += 1
|
||||
if websocket.application_state != WebSocketState.CONNECTED:
|
||||
break
|
||||
msg.special_kwargs['uuid'] = uuid.uuid4().hex
|
||||
print(msg)
|
||||
await websocket.send_bytes(msg.model_dump_json())
|
||||
except Exception as e:
|
||||
logger.exception(f"Error in wait_message_to_send: {e}")
|
||||
raise e
|
||||
|
||||
async def receive_forever(queue_realtime_from_client: asyncio.Queue, queue_blocking_from_client: asyncio.Queue, queue_back_to_client: asyncio.Queue, terminate_event: asyncio.Event):
|
||||
"""
|
||||
永久接收客户端消息
|
||||
|
||||
持续监听WebSocket连接,接收客户端消息并根据消息类型分发到不同队列。
|
||||
|
||||
参数:
|
||||
queue_realtime_from_client: 实时消息队列
|
||||
queue_blocking_from_client: 阻塞消息队列
|
||||
queue_back_to_client: 发送回客户端的消息队列
|
||||
"""
|
||||
# 🕜 keep listening traffic <- front end
|
||||
msg_cnt:int = 0
|
||||
try:
|
||||
while True:
|
||||
################
|
||||
# get message and check terminate
|
||||
while True:
|
||||
try:
|
||||
if terminate_event.is_set():
|
||||
msg = TERMINATE_MSG
|
||||
break
|
||||
else:
|
||||
message = await asyncio.wait_for(websocket.receive_text(), timeout=0.25)
|
||||
msg: UserInterfaceMsg = UserInterfaceMsg.model_validate_json(message)
|
||||
break
|
||||
except asyncio.TimeoutError:
|
||||
continue # 继续检查条件
|
||||
if msg.function == TERMINATE_MSG.function:
|
||||
logger.info("Received terminate message, stopping receive_forever")
|
||||
break
|
||||
################
|
||||
msg_cnt += 1
|
||||
logger.info(f"Received message {msg_cnt}: {msg}")
|
||||
queue_blocking_from_client.put_nowait(msg)
|
||||
|
||||
|
||||
except Exception as e:
|
||||
logger.exception(f"Error in receive_forever: {e}")
|
||||
raise e
|
||||
|
||||
queue_back_to_client, queue_realtime_from_client, queue_blocking_from_client, terminate_event = self.make_queue()
|
||||
|
||||
def terminate_callback():
|
||||
terminate_event.set()
|
||||
|
||||
def ensure_last_message_sent(queue_back_to_client):
|
||||
"""
|
||||
确保最后一条消息被发送到客户端
|
||||
"""
|
||||
queue_back_to_client.put_nowait(TERMINATE_MSG)
|
||||
tic = time.time()
|
||||
termination_timeout = 5 # seconds
|
||||
while not queue_back_to_client.empty():
|
||||
time.sleep(0.25)
|
||||
if queue_back_to_client.empty():
|
||||
break
|
||||
if time.time() - tic > termination_timeout:
|
||||
break
|
||||
|
||||
def task_wrapper(func):
|
||||
def wrapper(*args, **kwargs):
|
||||
res = func(*args, **kwargs)
|
||||
ensure_last_message_sent(queue_back_to_client)
|
||||
terminate_callback()
|
||||
return res
|
||||
return wrapper
|
||||
|
||||
t_x = asyncio.create_task(wait_message_to_send(queue_back_to_client, terminate_event))
|
||||
t_r = asyncio.create_task(receive_forever(queue_realtime_from_client, queue_blocking_from_client, queue_back_to_client, terminate_event))
|
||||
task_thread = threading.Thread(target=task_wrapper(task_executor), args=(initial_msg, queue_blocking_from_client, queue_back_to_client), daemon=True)
|
||||
task_thread.start()
|
||||
|
||||
await t_x
|
||||
await t_r
|
||||
await websocket.close()
|
||||
|
||||
|
||||
|
||||
class MasterMindWebSocketServer(PythonMethod_AsyncConnectionMaintainer_AgentcraftInterface):
|
||||
"""
|
||||
WebSocket服务器主类
|
||||
|
||||
继承自异步连接维护器接口,实现了完整的WebSocket服务器功能。
|
||||
负责处理客户端连接、事件管理和消息路由。
|
||||
"""
|
||||
|
||||
def __init__(self, host, port) -> None:
|
||||
"""
|
||||
初始化WebSocket服务器
|
||||
|
||||
参数:
|
||||
host: 服务器主机地址
|
||||
port: 服务器端口号
|
||||
"""
|
||||
self.websocket_connections = {}
|
||||
self.agentcraft_interface_websocket_connections = {}
|
||||
self._event_hub = {}
|
||||
self.host= host
|
||||
self.port = port
|
||||
pass
|
||||
|
||||
def create_event(self, event_name: str):
|
||||
"""
|
||||
创建一个新的事件
|
||||
|
||||
参数:
|
||||
event_name: 事件名称
|
||||
|
||||
返回:
|
||||
FutureEvent: 新创建的事件对象
|
||||
"""
|
||||
self._event_hub[event_name] = FutureEvent()
|
||||
return self._event_hub[event_name]
|
||||
|
||||
def terminate_event(self, event_name: str, msg:UserInterfaceMsg):
|
||||
"""
|
||||
终止指定的事件并设置返回消息
|
||||
|
||||
参数:
|
||||
event_name: 要终止的事件名称
|
||||
msg: 要返回的用户界面消息
|
||||
"""
|
||||
self._event_hub[event_name].terminate(return_value = msg)
|
||||
return
|
||||
|
||||
async def long_task_01_wait_incoming_connection(self):
|
||||
"""
|
||||
等待传入连接的长期任务
|
||||
|
||||
启动FastAPI服务器并等待WebSocket连接。
|
||||
处理新的连接请求,建立WebSocket通信。
|
||||
"""
|
||||
# task 1 wait incoming agent connection
|
||||
logger.info("task 1 wait incoming agent connection")
|
||||
|
||||
async def launch_websocket_server():
|
||||
"""
|
||||
启动WebSocket服务器
|
||||
|
||||
创建FastAPI应用并配置WebSocket路由,
|
||||
设置服务器参数并启动uvicorn服务器。
|
||||
"""
|
||||
app = FastAPI()
|
||||
|
||||
class UserInput(BaseModel):
|
||||
main_input: str
|
||||
@app.post("/predict_user_input")
|
||||
async def predict_user_input(main_input: UserInput):
|
||||
def predict_future_input(main_input):
|
||||
from request_llms.bridge_all import predict_no_ui_long_connection
|
||||
from toolbox import get_plugin_default_kwargs
|
||||
from textwrap import dedent
|
||||
com = get_plugin_default_kwargs()
|
||||
llm_kwargs = com['llm_kwargs']
|
||||
llm_kwargs['llm_model'] = 'one-api-glm-4-flash'
|
||||
completion_system_prompt = dedent("""Predict the next input that the user might type.
|
||||
1. Do not repeat the <current_input>. The <future_input> should be a continuation of <current_input>.
|
||||
2. Use same language as the input.
|
||||
3. Do not predict too far ahead.
|
||||
4. Use punctuation when needed. Sometimes <future_input> has to begin with some punctuation.
|
||||
|
||||
Example:
|
||||
<current_input>北京发布暴</current_input>
|
||||
<future_input>雨红色预警</future_input>
|
||||
|
||||
<current_input>世界是你们的,也是我们</current_input>
|
||||
<future_input>的,但是归根结底是你们的</future_input>
|
||||
|
||||
<current_input>多年以后,面对行刑队</current_input>
|
||||
<future_input>,奥雷里亚诺·布恩迪亚上校将会回想起父亲带他去见识冰块的那个遥远的下午。</future_input>
|
||||
|
||||
Format:
|
||||
<future_input>... the predicted input ...</future_input>
|
||||
""")
|
||||
main_input = "<current_input>" + main_input + "</current_input>"
|
||||
result = predict_no_ui_long_connection(
|
||||
inputs=main_input, llm_kwargs=llm_kwargs, sys_prompt=completion_system_prompt, history=[], console_silence=True
|
||||
)
|
||||
print(result)
|
||||
if "<future_input>" not in result or "</future_input>" not in result:
|
||||
raise ValueError("The response does not contain the expected future input format.")
|
||||
result = result.split("<future_input>")[-1].split("</future_input>")[0].strip()
|
||||
return result
|
||||
|
||||
return JSONResponse(content={'future':predict_future_input(main_input.main_input)})
|
||||
|
||||
@app.post("/core_functional")
|
||||
async def core_functional():
|
||||
import core_functional, importlib
|
||||
importlib.reload(core_functional) # 热更新prompt
|
||||
core_functionals = core_functional.get_core_functions()
|
||||
for k in list(core_functionals.keys()):
|
||||
v = core_functionals[k]
|
||||
# remove PreProcess if any
|
||||
if "PreProcess" in v:
|
||||
v.pop("PreProcess")
|
||||
if "Visible" in v and not v["Visible"]:
|
||||
core_functionals.pop(k)
|
||||
return JSONResponse(content=core_functionals)
|
||||
|
||||
@app.post("/upload")
|
||||
async def upload_files(files: List[UploadFile] = File(...)):
|
||||
"""上传文件接口,支持多文件上传并显示进度"""
|
||||
from toolbox import on_file_uploaded
|
||||
results = []
|
||||
upload_dir = "uploads"
|
||||
|
||||
# 确保上传目录存在
|
||||
if not os.path.exists(upload_dir):
|
||||
os.makedirs(upload_dir)
|
||||
|
||||
for file in files:
|
||||
try:
|
||||
# 构建文件保存路径
|
||||
file_path = os.path.join(upload_dir, file.filename)
|
||||
total_size = 0
|
||||
processed_size = 0
|
||||
|
||||
# 分块读取并保存文件,同时计算进度
|
||||
with open(file_path, "wb") as buffer:
|
||||
while chunk := await file.read(8192): # 8KB chunks
|
||||
buffer.write(chunk)
|
||||
processed_size += len(chunk)
|
||||
if file.size: # 如果文件大小可用
|
||||
progress = (processed_size / file.size) * 100
|
||||
logger.info(f"Uploading {file.filename}: {progress:.2f}%")
|
||||
|
||||
results.append({
|
||||
"filename": file.filename,
|
||||
"size": processed_size,
|
||||
"status": "success",
|
||||
"path": file_path
|
||||
})
|
||||
except Exception as e:
|
||||
results.append({
|
||||
"filename": file.filename,
|
||||
"status": "error",
|
||||
"error": str(e)
|
||||
})
|
||||
logger.error(f"Error uploading {file.filename}: {str(e)}")
|
||||
|
||||
return JSONResponse(content={
|
||||
"message": "Files uploaded successfully",
|
||||
"files": results
|
||||
})
|
||||
|
||||
@app.websocket("/main")
|
||||
async def main(websocket: WebSocket):
|
||||
"""
|
||||
WebSocket连接的主处理函数
|
||||
|
||||
处理新的WebSocket连接请求,接收初始消息并建立持久连接。
|
||||
|
||||
参数:
|
||||
websocket: WebSocket连接对象
|
||||
"""
|
||||
try:
|
||||
await websocket.accept()
|
||||
logger.info(f"WebSocket connection established: {websocket.client.host}:{websocket.client.port}")
|
||||
msg: UserInterfaceMsg = UserInterfaceMsg.model_validate_json(await websocket.receive_text())
|
||||
logger.info(msg)
|
||||
await self.maintain_connection_forever(msg, websocket, "client_id")
|
||||
except:
|
||||
logger.exception("Error in WebSocket connection handler")
|
||||
await websocket.close()
|
||||
import uvicorn
|
||||
config = uvicorn.Config(app, host=self.host, port=self.port, log_level="error", ws_ping_interval=300, ws_ping_timeout=600)
|
||||
server = uvicorn.Server(config)
|
||||
logger.info(f"uvicorn begin, serving at ws://{self.host}:{self.port}/main")
|
||||
await server.serve()
|
||||
|
||||
await launch_websocket_server()
|
||||
logger.info("uvicorn terminated")
|
||||
|
||||
if __name__ == "__main__":
|
||||
mmwss = MasterMindWebSocketServer(host="0.0.0.0", port=38000)
|
||||
asyncio.run(mmwss.long_task_01_wait_incoming_connection())
|
||||
@@ -12,7 +12,7 @@ if __name__ == "__main__":
|
||||
|
||||
# plugin_test(plugin='crazy_functions.Internet_GPT->连接网络回答问题', main_input="谁是应急食品?")
|
||||
|
||||
# plugin_test(plugin='crazy_functions.函数动态生成->函数动态生成', main_input='交换图像的蓝色通道和红色通道', advanced_arg={"file_path_arg": "./build/ants.jpg"})
|
||||
# plugin_test(plugin='crazy_functions.Dynamic_Function_Generate->Dynamic_Function_Generate', main_input='交换图像的蓝色通道和红色通道', advanced_arg={"file_path_arg": "./build/ants.jpg"})
|
||||
|
||||
# plugin_test(plugin='crazy_functions.Latex_Function->Latex翻译中文并重新编译PDF', main_input="2307.07522")
|
||||
|
||||
@@ -23,13 +23,13 @@ if __name__ == "__main__":
|
||||
# main_input="G:/SEAFILE_LOCAL/50503047/我的资料库/学位/paperlatex/aaai/Fu_8368_with_appendix",
|
||||
# )
|
||||
|
||||
# plugin_test(plugin='crazy_functions.虚空终端->虚空终端', main_input='修改api-key为sk-jhoejriotherjep')
|
||||
# plugin_test(plugin='crazy_functions.Void_Terminal->Void_Terminal', main_input='修改api-key为sk-jhoejriotherjep')
|
||||
|
||||
# plugin_test(plugin='crazy_functions.批量翻译PDF文档_NOUGAT->批量翻译PDF文档', main_input='crazy_functions/test_project/pdf_and_word/aaai.pdf')
|
||||
# plugin_test(plugin='crazy_functions.PDF_Translate_Nougat->批量翻译PDF文档', main_input='crazy_functions/test_project/pdf_and_word/aaai.pdf')
|
||||
|
||||
# plugin_test(plugin='crazy_functions.虚空终端->虚空终端', main_input='调用插件,对C:/Users/fuqingxu/Desktop/旧文件/gpt/chatgpt_academic/crazy_functions/latex_fns中的python文件进行解析')
|
||||
# plugin_test(plugin='crazy_functions.Void_Terminal->Void_Terminal', main_input='调用插件,对C:/Users/fuqingxu/Desktop/旧文件/gpt/chatgpt_academic/crazy_functions/latex_fns中的python文件进行解析')
|
||||
|
||||
# plugin_test(plugin='crazy_functions.命令行助手->命令行助手', main_input='查看当前的docker容器列表')
|
||||
# plugin_test(plugin='crazy_functions.Commandline_Assistant->Commandline_Assistant', main_input='查看当前的docker容器列表')
|
||||
|
||||
# plugin_test(plugin='crazy_functions.SourceCode_Analyse->解析一个Python项目', main_input="crazy_functions/test_project/python/dqn")
|
||||
|
||||
@@ -41,15 +41,15 @@ if __name__ == "__main__":
|
||||
|
||||
# plugin_test(plugin='crazy_functions.PDF_Translate->批量翻译PDF文档', main_input='crazy_functions/test_project/pdf_and_word/aaai.pdf')
|
||||
|
||||
# plugin_test(plugin='crazy_functions.谷歌检索小助手->谷歌检索小助手', main_input="https://scholar.google.com/scholar?hl=en&as_sdt=0%2C5&q=auto+reinforcement+learning&btnG=")
|
||||
# plugin_test(plugin='crazy_functions.Google_Scholar_Assistant_Legacy->Google_Scholar_Assistant_Legacy', main_input="https://scholar.google.com/scholar?hl=en&as_sdt=0%2C5&q=auto+reinforcement+learning&btnG=")
|
||||
|
||||
# plugin_test(plugin='crazy_functions.总结word文档->总结word文档', main_input="crazy_functions/test_project/pdf_and_word")
|
||||
# plugin_test(plugin='crazy_functions.Word_Summary->Word_Summary', main_input="crazy_functions/test_project/pdf_and_word")
|
||||
|
||||
# plugin_test(plugin='crazy_functions.下载arxiv论文翻译摘要->下载arxiv论文并翻译摘要', main_input="1812.10695")
|
||||
# plugin_test(plugin='crazy_functions.Arxiv_Downloader->下载arxiv论文并翻译摘要', main_input="1812.10695")
|
||||
|
||||
# plugin_test(plugin='crazy_functions.解析JupyterNotebook->解析ipynb文件', main_input="crazy_functions/test_samples")
|
||||
# plugin_test(plugin='crazy_functions.SourceCode_Analyse_JupyterNotebook->解析ipynb文件', main_input="crazy_functions/test_samples")
|
||||
|
||||
# plugin_test(plugin='crazy_functions.数学动画生成manim->动画生成', main_input="A ball split into 2, and then split into 4, and finally split into 8.")
|
||||
# plugin_test(plugin='crazy_functions.Math_Animation_Gen->动画生成', main_input="A ball split into 2, and then split into 4, and finally split into 8.")
|
||||
|
||||
# for lang in ["English", "French", "Japanese", "Korean", "Russian", "Italian", "German", "Portuguese", "Arabic"]:
|
||||
# plugin_test(plugin='crazy_functions.Markdown_Translate->Markdown翻译指定语言', main_input="README.md", advanced_arg={"advanced_arg": lang})
|
||||
|
||||
@@ -9,4 +9,4 @@ import os, sys
|
||||
if __name__ == "__main__":
|
||||
from test_utils import plugin_test
|
||||
|
||||
plugin_test(plugin='crazy_functions.数学动画生成manim->动画生成', main_input="A point moving along function culve y=sin(x), starting from x=0 and stop at x=4*\pi.")
|
||||
plugin_test(plugin='crazy_functions.Math_Animation_Gen->动画生成', main_input="A point moving along function culve y=sin(x), starting from x=0 and stop at x=4*\pi.")
|
||||
|
||||
@@ -13,7 +13,7 @@ if __name__ == "__main__":
|
||||
|
||||
# plugin_test(plugin='crazy_functions.Internet_GPT->连接网络回答问题', main_input="谁是应急食品?")
|
||||
|
||||
# plugin_test(plugin='crazy_functions.函数动态生成->函数动态生成', main_input='交换图像的蓝色通道和红色通道', advanced_arg={"file_path_arg": "./build/ants.jpg"})
|
||||
# plugin_test(plugin='crazy_functions.Dynamic_Function_Generate->Dynamic_Function_Generate', main_input='交换图像的蓝色通道和红色通道', advanced_arg={"file_path_arg": "./build/ants.jpg"})
|
||||
|
||||
# plugin_test(plugin='crazy_functions.Latex_Function->Latex翻译中文并重新编译PDF', main_input="2307.07522")
|
||||
|
||||
@@ -24,13 +24,13 @@ if __name__ == "__main__":
|
||||
# main_input="G:/SEAFILE_LOCAL/50503047/我的资料库/学位/paperlatex/aaai/Fu_8368_with_appendix",
|
||||
# )
|
||||
|
||||
# plugin_test(plugin='crazy_functions.虚空终端->虚空终端', main_input='修改api-key为sk-jhoejriotherjep')
|
||||
# plugin_test(plugin='crazy_functions.Void_Terminal->Void_Terminal', main_input='修改api-key为sk-jhoejriotherjep')
|
||||
|
||||
# plugin_test(plugin='crazy_functions.批量翻译PDF文档_NOUGAT->批量翻译PDF文档', main_input='crazy_functions/test_project/pdf_and_word/aaai.pdf')
|
||||
# plugin_test(plugin='crazy_functions.PDF_Translate_Nougat->批量翻译PDF文档', main_input='crazy_functions/test_project/pdf_and_word/aaai.pdf')
|
||||
|
||||
# plugin_test(plugin='crazy_functions.虚空终端->虚空终端', main_input='调用插件,对C:/Users/fuqingxu/Desktop/旧文件/gpt/chatgpt_academic/crazy_functions/latex_fns中的python文件进行解析')
|
||||
# plugin_test(plugin='crazy_functions.Void_Terminal->Void_Terminal', main_input='调用插件,对C:/Users/fuqingxu/Desktop/旧文件/gpt/chatgpt_academic/crazy_functions/latex_fns中的python文件进行解析')
|
||||
|
||||
# plugin_test(plugin='crazy_functions.命令行助手->命令行助手', main_input='查看当前的docker容器列表')
|
||||
# plugin_test(plugin='crazy_functions.Commandline_Assistant->Commandline_Assistant', main_input='查看当前的docker容器列表')
|
||||
|
||||
# plugin_test(plugin='crazy_functions.SourceCode_Analyse->解析一个Python项目', main_input="crazy_functions/test_project/python/dqn")
|
||||
|
||||
@@ -42,15 +42,15 @@ if __name__ == "__main__":
|
||||
|
||||
# plugin_test(plugin='crazy_functions.PDF_Translate->批量翻译PDF文档', main_input='crazy_functions/test_project/pdf_and_word/aaai.pdf')
|
||||
|
||||
# plugin_test(plugin='crazy_functions.谷歌检索小助手->谷歌检索小助手', main_input="https://scholar.google.com/scholar?hl=en&as_sdt=0%2C5&q=auto+reinforcement+learning&btnG=")
|
||||
# plugin_test(plugin='crazy_functions.Google_Scholar_Assistant_Legacy->Google_Scholar_Assistant_Legacy', main_input="https://scholar.google.com/scholar?hl=en&as_sdt=0%2C5&q=auto+reinforcement+learning&btnG=")
|
||||
|
||||
# plugin_test(plugin='crazy_functions.总结word文档->总结word文档', main_input="crazy_functions/test_project/pdf_and_word")
|
||||
# plugin_test(plugin='crazy_functions.Word_Summary->Word_Summary', main_input="crazy_functions/test_project/pdf_and_word")
|
||||
|
||||
# plugin_test(plugin='crazy_functions.下载arxiv论文翻译摘要->下载arxiv论文并翻译摘要', main_input="1812.10695")
|
||||
# plugin_test(plugin='crazy_functions.Arxiv_Downloader->下载arxiv论文并翻译摘要', main_input="1812.10695")
|
||||
|
||||
# plugin_test(plugin='crazy_functions.解析JupyterNotebook->解析ipynb文件', main_input="crazy_functions/test_samples")
|
||||
# plugin_test(plugin='crazy_functions.SourceCode_Analyse_JupyterNotebook->解析ipynb文件', main_input="crazy_functions/test_samples")
|
||||
|
||||
# plugin_test(plugin='crazy_functions.数学动画生成manim->动画生成', main_input="A ball split into 2, and then split into 4, and finally split into 8.")
|
||||
# plugin_test(plugin='crazy_functions.Math_Animation_Gen->动画生成', main_input="A ball split into 2, and then split into 4, and finally split into 8.")
|
||||
|
||||
# for lang in ["English", "French", "Japanese", "Korean", "Russian", "Italian", "German", "Portuguese", "Arabic"]:
|
||||
# plugin_test(plugin='crazy_functions.Markdown_Translate->Markdown翻译指定语言', main_input="README.md", advanced_arg={"advanced_arg": lang})
|
||||
|
||||
@@ -13,7 +13,7 @@ if __name__ == "__main__":
|
||||
|
||||
# plugin_test(plugin='crazy_functions.Internet_GPT->连接网络回答问题', main_input="谁是应急食品?")
|
||||
|
||||
# plugin_test(plugin='crazy_functions.函数动态生成->函数动态生成', main_input='交换图像的蓝色通道和红色通道', advanced_arg={"file_path_arg": "./build/ants.jpg"})
|
||||
# plugin_test(plugin='crazy_functions.Dynamic_Function_Generate->Dynamic_Function_Generate', main_input='交换图像的蓝色通道和红色通道', advanced_arg={"file_path_arg": "./build/ants.jpg"})
|
||||
|
||||
# plugin_test(plugin='crazy_functions.Latex_Function->Latex翻译中文并重新编译PDF', main_input="2307.07522")
|
||||
|
||||
@@ -24,13 +24,13 @@ if __name__ == "__main__":
|
||||
# main_input="G:/SEAFILE_LOCAL/50503047/我的资料库/学位/paperlatex/aaai/Fu_8368_with_appendix",
|
||||
# )
|
||||
|
||||
# plugin_test(plugin='crazy_functions.虚空终端->虚空终端', main_input='修改api-key为sk-jhoejriotherjep')
|
||||
# plugin_test(plugin='crazy_functions.Void_Terminal->Void_Terminal', main_input='修改api-key为sk-jhoejriotherjep')
|
||||
|
||||
# plugin_test(plugin='crazy_functions.批量翻译PDF文档_NOUGAT->批量翻译PDF文档', main_input='crazy_functions/test_project/pdf_and_word/aaai.pdf')
|
||||
# plugin_test(plugin='crazy_functions.PDF_Translate_Nougat->批量翻译PDF文档', main_input='crazy_functions/test_project/pdf_and_word/aaai.pdf')
|
||||
|
||||
# plugin_test(plugin='crazy_functions.虚空终端->虚空终端', main_input='调用插件,对C:/Users/fuqingxu/Desktop/旧文件/gpt/chatgpt_academic/crazy_functions/latex_fns中的python文件进行解析')
|
||||
# plugin_test(plugin='crazy_functions.Void_Terminal->Void_Terminal', main_input='调用插件,对C:/Users/fuqingxu/Desktop/旧文件/gpt/chatgpt_academic/crazy_functions/latex_fns中的python文件进行解析')
|
||||
|
||||
# plugin_test(plugin='crazy_functions.命令行助手->命令行助手', main_input='查看当前的docker容器列表')
|
||||
# plugin_test(plugin='crazy_functions.Commandline_Assistant->Commandline_Assistant', main_input='查看当前的docker容器列表')
|
||||
|
||||
# plugin_test(plugin='crazy_functions.SourceCode_Analyse->解析一个Python项目', main_input="crazy_functions/test_project/python/dqn")
|
||||
|
||||
@@ -42,15 +42,15 @@ if __name__ == "__main__":
|
||||
|
||||
# plugin_test(plugin='crazy_functions.PDF_Translate->批量翻译PDF文档', main_input='crazy_functions/test_project/pdf_and_word/aaai.pdf')
|
||||
|
||||
# plugin_test(plugin='crazy_functions.谷歌检索小助手->谷歌检索小助手', main_input="https://scholar.google.com/scholar?hl=en&as_sdt=0%2C5&q=auto+reinforcement+learning&btnG=")
|
||||
# plugin_test(plugin='crazy_functions.Google_Scholar_Assistant_Legacy->Google_Scholar_Assistant_Legacy', main_input="https://scholar.google.com/scholar?hl=en&as_sdt=0%2C5&q=auto+reinforcement+learning&btnG=")
|
||||
|
||||
# plugin_test(plugin='crazy_functions.总结word文档->总结word文档', main_input="crazy_functions/test_project/pdf_and_word")
|
||||
# plugin_test(plugin='crazy_functions.Word_Summary->Word_Summary', main_input="crazy_functions/test_project/pdf_and_word")
|
||||
|
||||
# plugin_test(plugin='crazy_functions.下载arxiv论文翻译摘要->下载arxiv论文并翻译摘要', main_input="1812.10695")
|
||||
# plugin_test(plugin='crazy_functions.Arxiv_Downloader->下载arxiv论文并翻译摘要', main_input="1812.10695")
|
||||
|
||||
# plugin_test(plugin='crazy_functions.解析JupyterNotebook->解析ipynb文件', main_input="crazy_functions/test_samples")
|
||||
# plugin_test(plugin='crazy_functions.SourceCode_Analyse_JupyterNotebook->解析ipynb文件', main_input="crazy_functions/test_samples")
|
||||
|
||||
# plugin_test(plugin='crazy_functions.数学动画生成manim->动画生成', main_input="A ball split into 2, and then split into 4, and finally split into 8.")
|
||||
# plugin_test(plugin='crazy_functions.Math_Animation_Gen->动画生成', main_input="A ball split into 2, and then split into 4, and finally split into 8.")
|
||||
|
||||
# for lang in ["English", "French", "Japanese", "Korean", "Russian", "Italian", "German", "Portuguese", "Arabic"]:
|
||||
# plugin_test(plugin='crazy_functions.Markdown_Translate->Markdown翻译指定语言', main_input="README.md", advanced_arg={"advanced_arg": lang})
|
||||
|
||||
@@ -18,11 +18,11 @@ validate_path() # 返回项目根路径
|
||||
if __name__ == "__main__":
|
||||
from tests.test_utils import plugin_test
|
||||
|
||||
plugin_test(plugin="crazy_functions.知识库问答->知识库文件注入", main_input="./README.md")
|
||||
plugin_test(plugin="crazy_functions.Vectorstore_QA->知识库文件注入", main_input="./README.md")
|
||||
|
||||
plugin_test(
|
||||
plugin="crazy_functions.知识库问答->读取知识库作答",
|
||||
plugin="crazy_functions.Vectorstore_QA->读取知识库作答",
|
||||
main_input="What is the installation method?",
|
||||
)
|
||||
|
||||
plugin_test(plugin="crazy_functions.知识库问答->读取知识库作答", main_input="远程云服务器部署?")
|
||||
plugin_test(plugin="crazy_functions.Vectorstore_QA->读取知识库作答", main_input="远程云服务器部署?")
|
||||
|
||||
在新工单中引用
屏蔽一个用户