Skip to content

DRILL-8548: Integrate Apache Ranger authorization for Drill #3056

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
shfshihuafeng wants to merge 1 commit into
base: master
Choose a base branch
from
Open

DRILL-8548: Integrate Apache Ranger authorization for Drill #3056

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
178 changes: 178 additions & 0 deletions distribution/pom.xml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -73,6 +73,53 @@
<artifactId>drill-common</artifactId>
<version>${project.version}</version>
</dependency>
<dependency>
<groupId>org.apache.drill</groupId>
<artifactId>drill-ranger-plugin</artifactId>

@cgivre cgivre Aug 23, 2026

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

These distribution changes are unconditional, for a feature that defaults to enabled: false.

Between this dependency block and the copy-ranger-plugin-isolated-deps execution below, every Drill tarball now carries Ranger 2.8.0 plus a complete second JAX-RS stack: jersey-client, jersey-common, jersey-server, jersey-hk2, jersey-media-json-jackson, jersey-entity-filtering (2.35), hk2-api/hk2-locator/hk2-utils, aopalliance-repackaged, osgi-resource-locator, and the javax.* JAX-RS/annotation/inject APIs — alongside the Jersey 3.1.9 that Drill's own REST server uses.

Two costs, both borne by every user regardless of whether they run Ranger:

  • Tarball size, on top of an already large distribution.
  • CVE surface and triage load. Jersey 2.35 and HK2 2.6.1 are pinned to old lines here. Every future advisory against them becomes something the Drill release manager has to answer for, even though the code is dormant in the default configuration.

Please put the whole thing behind a Maven profile (-Pranger, off by default), covering the dependency, the dependency-plugin execution, and the corresponding component.xml dependency sets. Operators who want Ranger opt in at build time; everyone else gets the current distribution unchanged.

The dual-Jersey coexistence via RangerPluginClassLoader is genuinely nice work and I don't think it's wrong — I'd just rather not ship both stacks to people who aren't using either one.

<version>${project.version}</version>
<exclusions>
<exclusion>
<groupId>org.glassfish.jersey.core</groupId>
<artifactId>*</artifactId>
</exclusion>
<exclusion>
<groupId>org.glassfish.jersey.inject</groupId>
<artifactId>*</artifactId>
</exclusion>
<exclusion>
<groupId>org.glassfish.hk2</groupId>
<artifactId>*</artifactId>
</exclusion>
<exclusion>
<groupId>org.glassfish.hk2.external</groupId>
<artifactId>*</artifactId>
</exclusion>
<exclusion>
<groupId>jakarta.ws.rs</groupId>
<artifactId>jakarta.ws.rs-api</artifactId>
</exclusion>
<exclusion>
<groupId>jakarta.annotation</groupId>
<artifactId>jakarta.annotation-api</artifactId>
</exclusion>
<exclusion>
<groupId>jakarta.inject</groupId>
<artifactId>jakarta.inject-api</artifactId>
</exclusion>
</exclusions>
</dependency>
<!--
Ranger Admin service plugin (REST-based, JDK 8 bytecode, no Drill deps).
Deployed into Ranger Admin's WEB-INF/lib. No shade/fat jar needed:
the plugin uses only HttpURLConnection (JDK-standard) and Jackson
(provided by Ranger Admin).
-->
<dependency>
<groupId>org.apache.drill</groupId>
<artifactId>ranger-drill-service</artifactId>
<version>${project.version}</version>
<type>jar</type>
</dependency>
<dependency>
<groupId>org.apache.drill</groupId>
<artifactId>drill-yarn</artifactId>
Expand Down Expand Up @@ -279,6 +326,137 @@

<build>
<plugins>
<plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-dependency-plugin</artifactId>
<executions>
<execution>
<id>copy-ranger-plugin-isolated-deps</id>
<phase>prepare-package</phase>
<goals>
<goal>copy</goal>
</goals>
<configuration>
<!--
Force-copy the 2.x versions of Jersey, HK2, JAX-RS API,
and annotation APIs that Maven dependency mediation would
otherwise resolve to 3.x (because Drill's own REST server
depends on the 3.x versions at a shallower depth).

These jars go into the isolated ranger-drill-plugin-impl/
directory and are loaded by RangerPluginClassLoader, NOT
the Drillbit classloader. Using maven-dependency-plugin:copy
(not the assembly dependencySet) ensures we get the exact
versions required by Jersey 2.35 / HK2 2.6.1, regardless of
what Maven mediation picks for the main classpath.
-->
<artifactItems>
<!-- JAX-RS 2.x API (javax.ws.rs.* package) -->
<artifactItem>
<groupId>jakarta.ws.rs</groupId>
<artifactId>jakarta.ws.rs-api</artifactId>
<version>${jaxrs.api.version}</version>
<type>jar</type>
<outputDirectory>${project.build.directory}/ranger-drill-plugin-impl</outputDirectory>
</artifactItem>
<!-- Jersey 2.x client runtime -->
<artifactItem>
<groupId>org.glassfish.jersey.core</groupId>
<artifactId>jersey-client</artifactId>
<version>${jersey.ranger.version}</version>
<type>jar</type>
<outputDirectory>${project.build.directory}/ranger-drill-plugin-impl</outputDirectory>
</artifactItem>
<artifactItem>
<groupId>org.glassfish.jersey.core</groupId>
<artifactId>jersey-common</artifactId>
<version>${jersey.ranger.version}</version>
<type>jar</type>
<outputDirectory>${project.build.directory}/ranger-drill-plugin-impl</outputDirectory>
</artifactItem>
<artifactItem>
<groupId>org.glassfish.jersey.inject</groupId>
<artifactId>jersey-hk2</artifactId>
<version>${jersey.ranger.version}</version>
<type>jar</type>
<outputDirectory>${project.build.directory}/ranger-drill-plugin-impl</outputDirectory>
</artifactItem>
<artifactItem>
<groupId>org.glassfish.jersey.core</groupId>
<artifactId>jersey-server</artifactId>
<version>${jersey.ranger.version}</version>
<type>jar</type>
<outputDirectory>${project.build.directory}/ranger-drill-plugin-impl</outputDirectory>
</artifactItem>
<artifactItem>
<groupId>org.glassfish.jersey.media</groupId>
<artifactId>jersey-media-json-jackson</artifactId>
<version>${jersey.ranger.version}</version>
<type>jar</type>
<outputDirectory>${project.build.directory}/ranger-drill-plugin-impl</outputDirectory>
</artifactItem>
<artifactItem>
<groupId>org.glassfish.jersey.ext</groupId>
<artifactId>jersey-entity-filtering</artifactId>
<version>${jersey.ranger.version}</version>
<type>jar</type>
<outputDirectory>${project.build.directory}/ranger-drill-plugin-impl</outputDirectory>
</artifactItem>
<!-- HK2 2.6.1 (DI container used by Jersey 2.x) -->
<artifactItem>
<groupId>org.glassfish.hk2</groupId>
<artifactId>hk2-locator</artifactId>
<version>${hk2.ranger.version}</version>
<type>jar</type>
<outputDirectory>${project.build.directory}/ranger-drill-plugin-impl</outputDirectory>
</artifactItem>
<artifactItem>
<groupId>org.glassfish.hk2</groupId>
<artifactId>hk2-api</artifactId>
<version>${hk2.ranger.version}</version>
<type>jar</type>
<outputDirectory>${project.build.directory}/ranger-drill-plugin-impl</outputDirectory>
</artifactItem>
<artifactItem>
<groupId>org.glassfish.hk2</groupId>
<artifactId>hk2-utils</artifactId>
<version>${hk2.ranger.version}</version>
<type>jar</type>
<outputDirectory>${project.build.directory}/ranger-drill-plugin-impl</outputDirectory>
</artifactItem>
<artifactItem>
<groupId>org.glassfish.hk2.external</groupId>
<artifactId>aopalliance-repackaged</artifactId>
<version>${hk2.ranger.version}</version>
<type>jar</type>
<outputDirectory>${project.build.directory}/ranger-drill-plugin-impl</outputDirectory>
</artifactItem>
<artifactItem>
<groupId>org.glassfish.hk2</groupId>
<artifactId>osgi-resource-locator</artifactId>
<version>${osgi.resource.locator.version}</version>
<type>jar</type>
<outputDirectory>${project.build.directory}/ranger-drill-plugin-impl</outputDirectory>
</artifactItem>
<artifactItem>
<groupId>jakarta.annotation</groupId>
<artifactId>jakarta.annotation-api</artifactId>
<version>${jakarta.annotation.api.version}</version>
<type>jar</type>
<outputDirectory>${project.build.directory}/ranger-drill-plugin-impl</outputDirectory>
</artifactItem>
<artifactItem>
<groupId>jakarta.inject</groupId>
<artifactId>jakarta.inject-api</artifactId>
<version>${jakarta.inject.api.version}</version>
<type>jar</type>
<outputDirectory>${project.build.directory}/ranger-drill-plugin-impl</outputDirectory>
</artifactItem>
</artifactItems>
</configuration>
</execution>
</executions>
</plugin>
<plugin>
<artifactId>maven-assembly-plugin</artifactId>
<executions>
Expand Down
104 changes: 102 additions & 2 deletions distribution/src/assemble/component.xml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -98,6 +98,68 @@
<useProjectArtifact>false</useProjectArtifact>
</dependencySet>

<!--
Ranger Admin service plugin. Kept in its own directory so it never lands on the
Drillbit classpath. Users copy this jar into Ranger Admin's
WEB-INF/classes/lib to enable Test Connection and resource autocomplete
in the Ranger policy editor.
-->
<dependencySet>
<includes>
<include>org.apache.drill:ranger-drill-service:jar</include>
</includes>
<outputDirectory>jars/ranger-service</outputDirectory>
<useProjectArtifact>false</useProjectArtifact>
</dependencySet>

<!--
Ranger framework jars (NOT Jersey/HK2 — those are force-copied by
maven-dependency-plugin:copy in distribution/pom.xml to ensure
correct 2.x versions are used).

These jars are NOT on the Drillbit's main classpath; they are loaded
by RangerPluginClassLoader from this directory. This isolation is
required because the plugin ships Jersey 2.35 (org.glassfish.jersey.*
+ javax.ws.rs.*) which conflicts with Drill's own Jersey 3.1.9
(org.glassfish.jersey.* + jakarta.ws.rs.*) on the main classpath.

Shared deps (SLF4J, Netty, Hadoop, Jackson) are 'provided' scope in
drill-ranger-plugin/pom.xml and intentionally NOT listed here — the
plugin classloader falls back to the Drillbit classloader for those.
-->
<dependencySet>
<includes>
<!-- Plugin itself -->
<include>org.apache.drill:drill-ranger-plugin:jar</include>
<!-- Ranger plugin framework -->
<include>org.apache.ranger:ranger-plugins-common:jar</include>
<include>org.apache.ranger:ranger-knox-plugin:jar</include>
<include>org.apache.ranger:ranger-audit-core:jar</include>
<include>org.apache.ranger:ranger-audit-dest-log4j:jar</include>
<include>org.apache.ranger:ranger-authz-api:jar</include>
<include>org.apache.ranger:ugsync-util:jar</include>
</includes>
<outputDirectory>jars/ranger-drill-plugin-impl</outputDirectory>
<unpack>false</unpack>
<useProjectArtifact>false</useProjectArtifact>
</dependencySet>

<!--
RangerPluginClassLoader must be on the Drillbit's main classpath because
RangerAccessAuthorizer (in drill-java-exec.jar) directly references it to
create the isolated classloader for the Ranger plugin. It cannot itself
be loaded by the isolated classloader.
-->
<dependencySet>
<includes>
<include>org.apache.ranger:ranger-plugin-classloader:jar</include>
</includes>
<outputDirectory>jars/3rdparty</outputDirectory>
<unpack>false</unpack>
<useProjectArtifact>false</useProjectArtifact>
<scope>compile</scope>
</dependencySet>

<dependencySet>
<outputDirectory>jars/classb</outputDirectory>
<unpack>false</unpack>
Expand Down Expand Up @@ -125,6 +187,22 @@
<include>org.jvnet.mimepull</include>
<include>org.reflections</include>
</includes>
<!--
Exclude Jersey 2.x jars (used by drill-ranger-plugin via
RangerPluginClassLoader). These go to jars/ranger-drill-plugin-impl/
instead, to avoid org.glassfish.jersey.* package conflicts with
Drill's own Jersey 3.1.9 on the main classpath.
-->
<excludes>
<exclude>org.glassfish.jersey.core:jersey-client:jar:${jersey.ranger.version}</exclude>
<exclude>org.glassfish.jersey.core:jersey-common:jar:${jersey.ranger.version}</exclude>
<exclude>org.glassfish.jersey.core:jersey-server:jar:${jersey.ranger.version}</exclude>
<exclude>org.glassfish.jersey.inject:jersey-hk2:jar:${jersey.ranger.version}</exclude>
<exclude>org.glassfish.jersey.media:jersey-media-json-jackson:jar:${jersey.ranger.version}</exclude>
<exclude>org.glassfish.jersey.media:jersey-media-multipart:jar:${jersey.ranger.version}</exclude>
<exclude>org.glassfish.jersey.ext:jersey-entity-filtering:jar:${jersey.ranger.version}</exclude>
<exclude>jakarta.ws.rs:jakarta.ws.rs-api:jar:${jaxrs.api.version}</exclude>
</excludes>
</dependencySet>
<dependencySet>
<outputDirectory>jars/3rdparty/</outputDirectory>
Expand Down Expand Up @@ -152,10 +230,16 @@
<exclude>org.apache.drill.exec</exclude>
<exclude>org.apache.drill.memory</exclude>
<exclude>org.apache.drill.metastore</exclude>
<!--
Ranger plugin jars and their transitive deps go to
jars/ranger-drill-plugin-impl/ (loaded by RangerPluginClassLoader),
NOT jars/3rdparty/ (main classpath).
-->
<exclude>org.apache.ranger</exclude>
<exclude>org.glassfish.hk2.external</exclude>
<exclude>org.glassfish.hk2</exclude>
<exclude>org.apache.zookeeper</exclude>
<exclude>org.eclipse.jetty</exclude>
<exclude>org.glassfish.hk2</exclude>
<exclude>org.glassfish.hk2.external</exclude>
<exclude>org.glassfish.jersey.containers</exclude>
<exclude>org.glassfish.jersey.core</exclude>
<exclude>org.glassfish.jersey.ext</exclude>
Expand Down Expand Up @@ -216,10 +300,26 @@
<directory>../sample-data</directory>
<outputDirectory>sample-data</outputDirectory>
</fileSet>
<!--
JAX-RS 2.x API (jakarta.ws.rs-api:2.1.6) and Jersey HK2 2.35 that were
force-copied by maven-dependency-plugin:copy in distribution/pom.xml.
These must use the 2.x versions, not the 3.x versions that Maven
dependency mediation would otherwise pick (because Drill's own REST
server depends on the 3.x versions at a shallower depth).
-->
<fileSet>
<directory>${project.build.directory}/ranger-drill-plugin-impl</directory>
<outputDirectory>jars/ranger-drill-plugin-impl</outputDirectory>
</fileSet>
<fileSet>
<directory>${project.build.directory}/winutils</directory>
<outputDirectory>winutils/bin</outputDirectory>
</fileSet>
<fileSet>
<directory>src/main/resources/ranger</directory>
<outputDirectory>conf/ranger</outputDirectory>
<fileMode>0640</fileMode>
</fileSet>
</fileSets>

<files>
Expand Down
5 changes: 5 additions & 0 deletions distribution/src/main/resources/drill-config.sh
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -365,6 +365,11 @@ export DRILLBIT_LOG_PATH="${DRILL_LOG_PREFIX}.log"
# Add Drill conf folder at the beginning of the classpath
CP="$DRILL_CONF_DIR"

# Add Ranger config directory if it exists (for ranger-drill-security.xml etc.)
if [ -d "$DRILL_CONF_DIR/ranger" ]; then
CP="$CP:$DRILL_CONF_DIR/ranger"
fi
Comment on lines +368 to +371

@cgivre cgivre Aug 23, 2026

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Guarding on directory existence is the right instinct, so this is harmless in practice — but it does prepend to CP for every Drillbit whether or not Ranger is enabled, and it's placed before the "Add Drill conf folder at the beginning of the classpath" block's intent is complete, so conf/ranger ends up ahead of some entries an operator might expect to win.

Two small things:

  • RangerAuthorization.md §3.3 Step 2 tells the operator to cp ranger-drill-security.xml $DRILL_HOME/conf/ (into conf/, not conf/ranger/), but §3.5 then refers to $DRILL_HOME/conf/ranger/ranger-drill-audit.xml. This code only adds conf/ranger. Following the doc as written produces a Drillbit that starts with Ranger enabled and no policy config on the classpath — which, given RangerBasePlugin denies by default with no policies, fails closed but with a confusing error. Please make the doc and the script agree on one location.
  • Consider gating on the same switch as everything else, e.g. only extend CP when the directory exists and the operator has opted in, so a stale conf/ranger left over from an experiment can't affect a Drillbit running with Ranger off.


# If both user and YARN-provided Java lib paths exist,
# combine them.

Expand Down
8 changes: 8 additions & 0 deletions distribution/src/main/resources/logback.xml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -73,6 +73,14 @@
<appender-ref ref="FILE" />
</logger>

<!-- Ranger audit logger: routes Log4jAuditProvider output into drillbit.log.
The logger name "xaaudit.org.apache.ranger.audit.provider.Log4jAuditProvider"
is constructed by Log4jAuditProvider static init as "xaaudit." + class.getName().
Audit events are logged at INFO level via AUDITLOG logger. -->
<logger name="xaaudit.org.apache.ranger.audit.provider.Log4jAuditProvider" additivity="false" level="info">
<appender-ref ref="FILE" />
</logger>

<logger name="query.logger" additivity="false" level="info">
<appender-ref ref="QUERY" />
<!-- <appender-ref ref="SOCKET" /> -->
Expand Down
Loading
Loading