Skip to content

Commit 5b25ce0

Browse files
dishaprakashgemini-code-assist[bot]averikitsch
authored
docs!: v1 docsite (#2831)
## Description This PR creates the new v1 docsite. ### Documentation Section - Clear use-case based folder structure and navigation ### Integrations Section - New folder structure for each source and compatible tools - Integration specific samples are moved into their folders ### Samples - New samples gallery to search for samples from all across the documentation site - Search and filter based on tags ### Reference - FAQ and CLI details ### Hugo - New secondary top navbar - Increased search capability ## Merge conflicts - versioning.md has been added to path `docs/en/reference/versioning.md` - style-guide has been added to path `docs/en/reference/style-guide.md` ## CloudFlare setup - workflows added to deploy to cloudflare for new release, dev and previous release - individual ephemeral PR preview using cloudflare pages ## Individual PRs - #2647 - #2648 - #2649 - #2650 - #2652 - #2664 - #2665 - #2666 - #2668 - #2670 - #2723 - #2779 - #2788 - #2790 ## PR Checklist > Thank you for opening a Pull Request! Before submitting your PR, there are a > few things you can do to make sure it goes smoothly: - [x] Make sure you reviewed [CONTRIBUTING.md](https://github.com/googleapis/genai-toolbox/blob/main/CONTRIBUTING.md) - [x] Make sure to open an issue as a [bug/issue](https://github.com/googleapis/genai-toolbox/issues/new/choose) before writing your code! That way we can discuss the change, evaluate designs, and agree on the general idea - [x] Ensure the tests and linter pass - [x] Code coverage does not decrease (if any source code was changed) - [x] Appropriate docs were updated (if necessary) - [x] Make sure to add `!` if this involve a breaking change 🛠️ Fixes #<issue_number_goes_here> --------- Co-authored-by: gemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com> Co-authored-by: Averi Kitsch <akitsch@google.com>
1 parent 5bef954 commit 5b25ce0

640 files changed

Lines changed: 7232 additions & 3883 deletions

File tree

Some content is hidden

Large Commits have some content hidden by default. Use the searchbox below for content that may be hidden.

‎.ci/lint-docs-source-page.sh‎

Lines changed: 136 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,136 @@
1+
#!/bin/bash
2+
set -e
3+
4+
5+
python3 - << 'EOF'
6+
"""
7+
MCP TOOLBOX: SOURCE PAGE LINTER
8+
===============================
9+
This script enforces a standardized structure for integration Source pages
10+
(source.md files). It ensures users can predictably find connection details
11+
and configurations across all database integrations.
12+
13+
Note: The structural _index.md folder wrappers are intentionally ignored
14+
by this script as they should only contain YAML frontmatter.
15+
16+
MAINTENANCE GUIDE:
17+
------------------
18+
1. TO ADD A NEW HEADING:
19+
Add the exact heading text to the 'ALLOWED_ORDER' list in the desired
20+
sequence.
21+
22+
2. TO MAKE A HEADING MANDATORY/OPTIONAL:
23+
Add or remove the heading text in the 'REQUIRED' set.
24+
25+
3. TO IGNORE NEW CONTENT TYPES:
26+
Update the regex in the 'clean_body' variable to strip out
27+
Markdown before linting.
28+
29+
4. SCOPE:
30+
This script only targets docs/en/integrations/**/source.md.
31+
"""
32+
33+
import os
34+
import re
35+
import sys
36+
from pathlib import Path
37+
38+
# --- CONFIGURATION ---
39+
ALLOWED_ORDER = [
40+
"About",
41+
"Available Tools",
42+
"Requirements",
43+
"Example",
44+
"Reference",
45+
"Advanced Usage",
46+
"Troubleshooting",
47+
"Additional Resources"
48+
]
49+
REQUIRED = {"About", "Example", "Reference"}
50+
51+
# Regex to catch any variation of the list-tools shortcode
52+
SHORTCODE_PATTERN = r"\{\{<\s*list-tools.*?>\}\}"
53+
# ---------------------
54+
55+
integration_dir = Path("./docs/en/integrations")
56+
if not integration_dir.exists():
57+
print("Info: Directory './docs/en/integrations' not found. Skipping linting.")
58+
sys.exit(0)
59+
60+
has_errors = False
61+
source_pages_found = 0
62+
63+
# ONLY scan files specifically named "source.md"
64+
for filepath in integration_dir.rglob("source.md"):
65+
source_pages_found += 1
66+
file_errors = False
67+
68+
if filepath.parent.parent != integration_dir:
69+
continue
70+
71+
with open(filepath, "r", encoding="utf-8") as f:
72+
content = f.read()
73+
74+
match = re.match(r'^\s*---\s*\n(.*?)\n---\s*(.*)', content, re.DOTALL)
75+
if match:
76+
frontmatter, body = match.group(1), match.group(2)
77+
else:
78+
print(f"[{filepath}] Error: Missing or invalid YAML frontmatter.")
79+
has_errors = True
80+
continue
81+
82+
# 1. Check for linkTitle: "Source" in frontmatter
83+
link_title_match = re.search(r"^linkTitle:\s*[\"']?(.*?)[\"']?\s*$", frontmatter, re.MULTILINE)
84+
if not link_title_match or link_title_match.group(1).strip() != "Source":
85+
print(f"[{filepath}] Error: Frontmatter must contain exactly linkTitle: \"Source\".")
86+
file_errors = True
87+
88+
# 2. Check for weight: 1 in frontmatter
89+
weight_match = re.search(r"^weight:\s*[\"']?(\d+)[\"']?\s*$", frontmatter, re.MULTILINE)
90+
if not weight_match or weight_match.group(1).strip() != "1":
91+
print(f"[{filepath}] Error: Frontmatter must contain exactly weight: 1.")
92+
file_errors = True
93+
94+
# 3. Check Shortcode Placement & Available Tools Section (Only if present)
95+
tools_section_match = re.search(r"^##\s+Available Tools\s*(.*?)(?=^##\s|\Z)", body, re.MULTILINE | re.DOTALL)
96+
if tools_section_match:
97+
if not re.search(SHORTCODE_PATTERN, tools_section_match.group(1)):
98+
print(f"[{filepath}] Error: The list-tools shortcode must be placed under the '## Available Tools' heading.")
99+
file_errors = True
100+
101+
# Strip code blocks from body to avoid linting example markdown headings
102+
clean_body = re.sub(r"```.*?```", "", body, flags=re.DOTALL)
103+
104+
if re.search(r"^#\s+\w+", clean_body, re.MULTILINE):
105+
print(f"[{filepath}] Error: H1 (#) headings are forbidden in the body.")
106+
file_errors = True
107+
108+
h2s = [h.strip() for h in re.findall(r"^##\s+(.*)", clean_body, re.MULTILINE)]
109+
110+
# Missing Required Headings
111+
missing = REQUIRED - set(h2s)
112+
if missing:
113+
print(f"[{filepath}] Error: Missing required H2 headings: {missing}")
114+
file_errors = True
115+
116+
if unauthorized := (set(h2s) - set(ALLOWED_ORDER)):
117+
print(f"[{filepath}] Error: Unauthorized H2s found: {unauthorized}")
118+
file_errors = True
119+
120+
# 5. Order Check
121+
if [h for h in h2s if h in ALLOWED_ORDER] != [h for h in ALLOWED_ORDER if h in h2s]:
122+
print(f"[{filepath}] Error: Headings out of order. Reference: {ALLOWED_ORDER}")
123+
file_errors = True
124+
125+
if file_errors: has_errors = True
126+
127+
# Handle final output based on what was found
128+
if source_pages_found == 0:
129+
print("Info: No 'source.md' files found in integrations. Passing gracefully.")
130+
sys.exit(0)
131+
elif has_errors:
132+
print(f"\nLinting failed. Please fix the structure errors in the {source_pages_found} 'source.md' file(s) above.")
133+
sys.exit(1)
134+
else:
135+
print(f"Success: {source_pages_found} 'source.md' file(s) passed structure validation.")
136+
EOF

‎.ci/lint-docs-tool-page.sh‎

Lines changed: 155 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,155 @@
1+
#!/bin/bash
2+
set -e
3+
4+
python3 - << 'EOF'
5+
"""
6+
MCP TOOLBOX: TOOL PAGE LINTER
7+
=============================
8+
This script enforces a standardized structure for individual Tool pages
9+
and their parent directory wrappers. It ensures LLM agents can parse
10+
tool capabilities and parameter definitions reliably.
11+
12+
MAINTENANCE GUIDE:
13+
------------------
14+
1. TO ADD A NEW HEADING:
15+
Add the exact heading text to the 'ALLOWED_ORDER' list in the desired
16+
sequence.
17+
18+
2. TO MAKE A HEADING MANDATORY/OPTIONAL:
19+
Add or remove the heading text in the 'REQUIRED' set.
20+
21+
3. TO UPDATE SHORTCODE LOGIC:
22+
If the shortcode name changes, update the 'SHORTCODE_PATTERN' variable.
23+
24+
4. SCOPE & BEHAVIOR:
25+
This script targets all .md files in docs/en/integrations/**/tools/.
26+
- For `_index.md` files: It only validates the frontmatter (requiring
27+
`title: "Tools"` and `weight: 2`) and ignores the body.
28+
- For regular tool files: It validates H1/H2 hierarchy, checks for
29+
required headings ("About", "Example"), and enforces that the
30+
`{{< compatible-sources >}}` shortcode is paired with the
31+
"## Compatible Sources" heading.
32+
"""
33+
34+
import os
35+
import re
36+
import sys
37+
from pathlib import Path
38+
39+
# --- CONFIGURATION ---
40+
ALLOWED_ORDER = [
41+
"About",
42+
"Compatible Sources",
43+
"Requirements",
44+
"Parameters",
45+
"Example",
46+
"Output Format",
47+
"Reference",
48+
"Advanced Usage",
49+
"Troubleshooting",
50+
"Additional Resources"
51+
]
52+
REQUIRED = {"About", "Example"}
53+
SHORTCODE_PATTERN = r"\{\{<\s*compatible-sources.*?>\}\}"
54+
# ---------------------
55+
56+
integration_dir = Path("./docs/en/integrations")
57+
if not integration_dir.exists():
58+
print("Info: Directory './docs/en/integrations' not found. Skipping linting.")
59+
sys.exit(0)
60+
61+
has_errors = False
62+
tools_pages_found = 0
63+
64+
# Specifically target the tools directories
65+
for filepath in integration_dir.rglob("tools/*.md"):
66+
tools_pages_found += 1
67+
with open(filepath, "r", encoding="utf-8") as f:
68+
content = f.read()
69+
70+
# Separate YAML frontmatter from the markdown body
71+
match = re.match(r'^\s*---\s*\n(.*?)\n---\s*(.*)', content, re.DOTALL)
72+
if match:
73+
frontmatter = match.group(1)
74+
body = match.group(2)
75+
else:
76+
print(f"[{filepath}] Error: Missing or invalid YAML frontmatter.")
77+
has_errors = True
78+
continue
79+
80+
file_errors = False
81+
82+
# --- SPECIAL VALIDATION FOR tools/_index.md ---
83+
if filepath.name == "_index.md":
84+
title_match = re.search(r"^title:\s*[\"']?(.*?)[\"']?\s*$", frontmatter, re.MULTILINE)
85+
if not title_match or title_match.group(1).strip() != "Tools":
86+
print(f"[{filepath}] Error: tools/_index.md must have exactly title: \"Tools\"")
87+
file_errors = True
88+
89+
weight_match = re.search(r"^weight:\s*(\d+)\s*$", frontmatter, re.MULTILINE)
90+
if not weight_match or weight_match.group(1).strip() != "2":
91+
print(f"[{filepath}] Error: tools/_index.md must have exactly weight: 2")
92+
file_errors = True
93+
94+
if file_errors:
95+
has_errors = True
96+
continue # Skip the rest of the body linting for this structural file
97+
98+
# --- VALIDATION FOR REGULAR TOOL PAGES ---
99+
# If the file has no markdown content (metadata placeholder only), skip it entirely
100+
if not body.strip():
101+
continue
102+
103+
# 1. Check Shortcode Placement
104+
sources_section_match = re.search(r"^##\s+Compatible Sources\s*(.*?)(?=^##\s|\Z)", body, re.MULTILINE | re.DOTALL)
105+
if sources_section_match:
106+
if not re.search(SHORTCODE_PATTERN, sources_section_match.group(1)):
107+
print(f"[{filepath}] Error: The compatible-sources shortcode must be placed under '## Compatible Sources'.")
108+
file_errors = True
109+
elif re.search(SHORTCODE_PATTERN, body):
110+
print(f"[{filepath}] Error: Shortcode found, but '## Compatible Sources' heading is missing.")
111+
file_errors = True
112+
113+
# 2. Strip code blocks from body to avoid linting example markdown headings
114+
clean_body = re.sub(r"```.*?```", "", body, flags=re.DOTALL)
115+
116+
# 3. Check H1 Headings
117+
if re.search(r"^#\s+\w+", clean_body, re.MULTILINE):
118+
print(f"[{filepath}] Error: H1 headings (#) are forbidden in the body.")
119+
file_errors = True
120+
121+
# 4. Check H2 Headings
122+
h2s = re.findall(r"^##\s+(.*)", clean_body, re.MULTILINE)
123+
h2s = [h2.strip() for h2 in h2s]
124+
125+
# Missing Required
126+
if missing := (REQUIRED - set(h2s)):
127+
print(f"[{filepath}] Error: Missing required H2 headings: {missing}")
128+
file_errors = True
129+
130+
# Unauthorized Headings
131+
if unauthorized := (set(h2s) - set(ALLOWED_ORDER)):
132+
print(f"[{filepath}] Error: Unauthorized H2 headings found: {unauthorized}")
133+
file_errors = True
134+
135+
# Strict Ordering
136+
filtered_h2s = [h for h in h2s if h in ALLOWED_ORDER]
137+
expected_order = [h for h in ALLOWED_ORDER if h in h2s]
138+
if filtered_h2s != expected_order:
139+
print(f"[{filepath}] Error: Headings are out of order.")
140+
print(f" Expected: {expected_order}")
141+
print(f" Found: {filtered_h2s}")
142+
file_errors = True
143+
144+
if file_errors:
145+
has_errors = True
146+
147+
if tools_pages_found == 0:
148+
print("Info: No tool directories found. Passing gracefully.")
149+
sys.exit(0)
150+
elif has_errors:
151+
print("Linting failed for Tool pages. Please fix the structure errors above.")
152+
sys.exit(1)
153+
else:
154+
print(f"Success: All {tools_pages_found} Tool page(s) passed structure validation.")
155+
EOF

‎.ci/sample_tests/pre_post_processing/go.integration.cloudbuild.yaml‎

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -48,7 +48,7 @@ timeout: 1200s
4848
substitutions:
4949
_TARGET_LANG: "go"
5050
_IMAGE: "golang:1.25.7"
51-
_TARGET_ROOT: "docs/en/samples/pre_post_processing/go"
51+
_TARGET_ROOT: "docs/en/documentation/configuration/pre-post-processing/go"
5252
_TABLE_NAME: "hotels_go_pre_post_processing"
5353
_SQL_FILE: ".ci/sample_tests/setup_hotels.sql"
5454
_AGENT_FILE_PATTERN: "agent.go"

‎.ci/sample_tests/pre_post_processing/js.integration.cloudbuild.yaml‎

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -48,7 +48,7 @@ timeout: 1200s
4848
substitutions:
4949
_TARGET_LANG: "js"
5050
_IMAGE: "node:22"
51-
_TARGET_ROOT: "docs/en/samples/pre_post_processing/js"
51+
_TARGET_ROOT: "docs/en/documentation/configuration/pre-post-processing/js"
5252
_TABLE_NAME: "hotels_js_pre_post_processing"
5353
_SQL_FILE: ".ci/sample_tests/setup_hotels.sql"
5454
_AGENT_FILE_PATTERN: "agent.js"

‎.ci/sample_tests/pre_post_processing/py.integration.cloudbuild.yaml‎

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -48,7 +48,7 @@ timeout: 1200s
4848
substitutions:
4949
_TARGET_LANG: "python"
5050
_IMAGE: "gcr.io/google.com/cloudsdktool/cloud-sdk:537.0.0"
51-
_TARGET_ROOT: "docs/en/samples/pre_post_processing/python"
51+
_TARGET_ROOT: "docs/en/documentation/configuration/pre-post-processing/python"
5252
_TABLE_NAME: "hotels_py_pre_post_processing"
5353
_SQL_FILE: ".ci/sample_tests/setup_hotels.sql"
5454
_AGENT_FILE_PATTERN: "agent.py"

‎.ci/sample_tests/quickstart/go.integration.cloudbuild.yaml‎

Lines changed: 3 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -13,7 +13,7 @@
1313
# limitations under the License.
1414

1515
steps:
16-
- name: 'golang:1.25.1'
16+
- name: 'golang:1.25.7'
1717
id: 'go-quickstart-test'
1818
entrypoint: 'bash'
1919
args:
@@ -30,7 +30,7 @@ steps:
3030
- 'GCP_PROJECT=${_GCP_PROJECT}'
3131
- 'DATABASE_NAME=${_DATABASE_NAME}'
3232
- 'DB_USER=${_DB_USER}'
33-
- 'TARGET_ROOT=docs/en/getting-started/quickstart/go'
33+
- 'TARGET_ROOT=docs/en/documentation/getting-started/quickstart/go'
3434
- 'TARGET_LANG=go'
3535
- 'TABLE_NAME=hotels_go'
3636
- 'SQL_FILE=.ci/sample_tests/setup_hotels.sql'
@@ -49,4 +49,4 @@ availableSecrets:
4949
timeout: 1000s
5050

5151
options:
52-
logging: CLOUD_LOGGING_ONLY
52+
logging: CLOUD_LOGGING_ONLY

‎.ci/sample_tests/quickstart/js.integration.cloudbuild.yaml‎

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -30,7 +30,7 @@ steps:
3030
- 'GCP_PROJECT=${_GCP_PROJECT}'
3131
- 'DATABASE_NAME=${_DATABASE_NAME}'
3232
- 'DB_USER=${_DB_USER}'
33-
- 'TARGET_ROOT=docs/en/getting-started/quickstart/js'
33+
- 'TARGET_ROOT=docs/en/documentation/getting-started/quickstart/js'
3434
- 'TARGET_LANG=js'
3535
- 'TABLE_NAME=hotels_js'
3636
- 'SQL_FILE=.ci/sample_tests/setup_hotels.sql'

‎.ci/sample_tests/quickstart/py.integration.cloudbuild.yaml‎

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -30,7 +30,7 @@ steps:
3030
- 'GCP_PROJECT=${_GCP_PROJECT}'
3131
- 'DATABASE_NAME=${_DATABASE_NAME}'
3232
- 'DB_USER=${_DB_USER}'
33-
- 'TARGET_ROOT=docs/en/getting-started/quickstart/python'
33+
- 'TARGET_ROOT=docs/en/documentation/getting-started/quickstart/python'
3434
- 'TARGET_LANG=python'
3535
- 'TABLE_NAME=hotels_python'
3636
- 'SQL_FILE=.ci/sample_tests/setup_hotels.sql'

‎.github/header-checker-lint.yml‎

Lines changed: 3 additions & 4 deletions
Original file line numberDiff line numberDiff line change
@@ -22,7 +22,6 @@ sourceFileExtensions:
2222
- 'yaml'
2323
- 'yml'
2424
ignoreFiles:
25-
- 'docs/en/getting-started/quickstart/**'
26-
- 'docs/en/samples/**'
27-
- '**/*oracle*'
28-
25+
- 'docs/en/documentation/**/*.go'
26+
- 'docs/en/samples/**/*'
27+
- '**/*oracle*'

0 commit comments

Comments
 (0)