Google Cloud Storage 是一种用于存储非结构化数据的托管服务。本文档介绍如何从
Google Cloud Storage (GCS) 目录(存储桶) 加载文档对象。
复制
向 AI 提问
pip install -qU langchain-google-community[gcs]
复制
向 AI 提问
from langchain_google_community import GCSDirectoryLoader
复制
向 AI 提问
loader = GCSDirectoryLoader(project_name="aist", bucket="testing-hwc")
复制
向 AI 提问
loader.load()
复制
向 AI 提问
/Users/harrisonchase/workplace/langchain/.venv/lib/python3.10/site-packages/google/auth/_default.py:83: UserWarning: Your application has authenticated using end user credentials from Google Cloud SDK without a quota project. You might receive a "quota exceeded" or "API not enabled" error. We recommend you rerun `gcloud auth application-default login` and make sure a quota project is added. Or you can use service accounts instead. For more information about service accounts, see https://cloud.google.com/docs/authentication/
warnings.warn(_CLOUD_SDK_CREDENTIALS_WARNING)
/Users/harrisonchase/workplace/langchain/.venv/lib/python3.10/site-packages/google/auth/_default.py:83: UserWarning: Your application has authenticated using end user credentials from Google Cloud SDK without a quota project. You might receive a "quota exceeded" or "API not enabled" error. We recommend you rerun `gcloud auth application-default login` and make sure a quota project is added. Or you can use service accounts instead. For more information about service accounts, see https://cloud.google.com/docs/authentication/
warnings.warn(_CLOUD_SDK_CREDENTIALS_WARNING)
复制
向 AI 提问
[Document(page_content='Lorem ipsum dolor sit amet.', lookup_str='', metadata={'source': '/var/folders/y6/8_bzdg295ld6s1_97_12m4lr0000gn/T/tmpz37njh7u/fake.docx'}, lookup_index=0)]
指定前缀
您还可以指定前缀,以便更精细地控制要加载的文件,包括从特定文件夹加载所有文件。复制
向 AI 提问
loader = GCSDirectoryLoader(project_name="aist", bucket="testing-hwc", prefix="fake")
复制
向 AI 提问
loader.load()
复制
向 AI 提问
/Users/harrisonchase/workplace/langchain/.venv/lib/python3.10/site-packages/google/auth/_default.py:83: UserWarning: Your application has authenticated using end user credentials from Google Cloud SDK without a quota project. You might receive a "quota exceeded" or "API not enabled" error. We recommend you rerun `gcloud auth application-default login` and make sure a quota project is added. Or you can use service accounts instead. For more information about service accounts, see https://cloud.google.com/docs/authentication/
warnings.warn(_CLOUD_SDK_CREDENTIALS_WARNING)
/Users/harrisonchase/workplace/langchain/.venv/lib/python3.10/site-packages/google/auth/_default.py:83: UserWarning: Your application has authenticated using end user credentials from Google Cloud SDK without a quota project. You might receive a "quota exceeded" or "API not enabled" error. We recommend you rerun `gcloud auth application-default login` and make sure a quota project is added. Or you can use service accounts instead. For more information about service accounts, see https://cloud.google.com/docs/authentication/
warnings.warn(_CLOUD_SDK_CREDENTIALS_WARNING)
复制
向 AI 提问
[Document(page_content='Lorem ipsum dolor sit amet.', lookup_str='', metadata={'source': '/var/folders/y6/8_bzdg295ld6s1_97_12m4lr0000gn/T/tmpylg6291i/fake.docx'}, lookup_index=0)]
加载单个文件失败时继续
GCS 存储桶中的文件在处理过程中可能会导致错误。启用continue_on_failure=True 参数以允许静默失败。这意味着处理单个文件失败不会导致函数中断,而是会记录警告。
复制
向 AI 提问
loader = GCSDirectoryLoader(
project_name="aist", bucket="testing-hwc", continue_on_failure=True
)
复制
向 AI 提问
loader.load()
以编程方式连接这些文档到 Claude、VSCode 等,通过 MCP 获取实时答案。