Support Request: Maintain cloud-sql-proxy.sh Compatibility with Dataproc 1.5 Lineage
Summary
Request to ensure the cloud-sql-proxy.sh initialization action retains robust compatibility with Dataproc 1.5 images (Debian 10 base), even as they move past End of Life (EOL). The primary goal is to guarantee that targeting a 1.5 base image continues to produce a fully working standalone Hive server configuration linked to an external metastore via the proxy.
Context
Legacy pipelines and existing custom images built on the Dataproc 1.5 lineage remain in production for extended periods. It is critical that the standard tooling for external metastore integration (cloud-sql-proxy.sh) can still successfully bootstrap these older environments without failing due to expired keys, archived repositories, or deprecated binary download paths.
Required Updates / Areas of Focus
-
Repository Maintenance: Ensure functions handling legacy repositories (e.g., repair_old_backports, clean_up_sources_lists) are maintained to handle archived Debian 10 repositories and expired GPG keys correctly.
-
Binary Availability: Ensure the process for retrieving cloud_sql_proxy binaries (V1 or V2) remains robust on older operating system cores.
-
Hive Integration: Guarantee that the configuration merges into hive-site.xml result in a functional Hive service talking to the external Cloud SQL instance.
The goal is to maintain this script as a reliable fallback or bridge for legacy environment maintenance.
Support Request: Maintain
cloud-sql-proxy.shCompatibility with Dataproc 1.5 LineageSummary
Request to ensure the
cloud-sql-proxy.shinitialization action retains robust compatibility with Dataproc 1.5 images (Debian 10 base), even as they move past End of Life (EOL). The primary goal is to guarantee that targeting a 1.5 base image continues to produce a fully working standalone Hive server configuration linked to an external metastore via the proxy.Context
Legacy pipelines and existing custom images built on the Dataproc 1.5 lineage remain in production for extended periods. It is critical that the standard tooling for external metastore integration (
cloud-sql-proxy.sh) can still successfully bootstrap these older environments without failing due to expired keys, archived repositories, or deprecated binary download paths.Required Updates / Areas of Focus
Repository Maintenance: Ensure functions handling legacy repositories (e.g.,
repair_old_backports,clean_up_sources_lists) are maintained to handle archived Debian 10 repositories and expired GPG keys correctly.Binary Availability: Ensure the process for retrieving
cloud_sql_proxybinaries (V1 or V2) remains robust on older operating system cores.Hive Integration: Guarantee that the configuration merges into
hive-site.xmlresult in a functional Hive service talking to the external Cloud SQL instance.The goal is to maintain this script as a reliable fallback or bridge for legacy environment maintenance.