Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
3 changes: 3 additions & 0 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -24,6 +24,9 @@ This project includes step-by-step guides for common datasource configuration sc
**See [docs/guides/README.md](docs/guides/README.md)** for:
- [Creating DataSource pools](docs/guides/create-datasource-pool.md) — basic pools, read-only pools, Kubernetes deployment, AWS Lambda, and configuration reference
- [Connection Validation Best Practices](docs/guides/connection-validation-best-practices.md) — how to configure heartbeat validation and why explicit heartbeatSql is rarely needed
- [Configuration Reference](docs/guides/configuration-reference.md) — every builder setting, its default, and the equivalent property key for external configuration
- [Troubleshooting Connection Leaks & Pool Exhaustion](docs/guides/troubleshooting-connection-leaks.md) — diagnosing pool exhaustion, leaks, and common warnings
- [Monitoring Pool Metrics](docs/guides/monitoring-pool-metrics.md) — reading PoolStatus metrics, pool state, and outage alerts
- [AWS Aurora with dual DataSources](docs/guides/aws-aurora-read-write-split.md) — read-write endpoint separation with Ebean ORM integration
- Instructions for AI agents to discover and follow these guides

Expand Down
3 changes: 3 additions & 0 deletions docs/guides/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -10,6 +10,9 @@ Step-by-step guides covering common scenarios for creating and configuring conne
|-------|-------------|
| [Create a DataSource Pool](create-datasource-pool.md) | Basic pool creation, read-only pools, Kubernetes configuration, AWS Lambda setup, and configuration reference |
| [Connection Validation Best Practices](connection-validation-best-practices.md) | How connection validation works; when and how to configure heartbeat validation; why explicit heartbeatSql is rarely needed |
| [Configuration Reference](configuration-reference.md) | Complete reference of every builder setting, its default, and the equivalent property key for file-based / external configuration |
| [Troubleshooting Connection Leaks & Pool Exhaustion](troubleshooting-connection-leaks.md) | Diagnose `ConnectionPoolExhaustedException`, connection leaks, captureStackTrace, leakTimeMinutes, and common warnings |
| [Monitoring Pool Metrics](monitoring-pool-metrics.md) | Read `PoolStatus` metrics, check pool state, and wire up outage alerts and borrow/return listeners |

## AWS Aurora

Expand Down
187 changes: 187 additions & 0 deletions docs/guides/configuration-reference.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,187 @@
# Guide: Configuration Reference

## Purpose

A complete reference of every `DataSourceBuilder` setting, its default value, and the equivalent
property key for file-based / external configuration. Use this alongside the task-focused guides
(pool creation, read-only pools, validation, Aurora) when you need to know exactly what a setting
does or what it defaults to.

---

## Two ways to configure

### 1. Programmatic (builder)

```java
DataSourcePool pool = DataSourcePool.builder()
.name("mypool")
.url("jdbc:postgresql://localhost:5432/myapp")
.username("app_user")
.password("password")
.minConnections(5)
.maxConnections(50)
.build();
```

### 2. Properties (external configuration)

The builder can load settings from `java.util.Properties`. There are three entry points:

```java
Properties props = ...; // loaded from file, env, avaje-config, Spring, etc.

// (a) no prefix: keys are "username", "url", ...
DataSourcePool pool = DataSourcePool.builder().load(props).build();

// (b) custom prefix: keys are "my-db.username", "my-db.url", ...
DataSourcePool pool = DataSourcePool.builder().load(props, "my-db").build();

// (c) "datasource.<poolName>." prefix: keys are "datasource.hr.username", ...
DataSourcePool pool = DataSourcePool.builder().loadSettings(props, "hr").build();
```

Example properties file using the `loadSettings` convention with pool name `hr`:

```properties
datasource.hr.username=app_user
datasource.hr.password=password
datasource.hr.url=jdbc:postgresql://localhost:5432/myapp
datasource.hr.minConnections=5
datasource.hr.maxConnections=50
datasource.hr.leakTimeMinutes=30
```

> When used through **Ebean ORM**, these `datasource.<name>.*` properties are typically placed in
> `application.yaml` / `application.properties` and loaded for you via avaje-config.

Property keys are matched case-insensitively.

---

## Connection settings

| Builder method | Property key | Default | Description |
|----------------|--------------|---------|-------------|
| `url(String)` | `url` (or `databaseUrl`) | – | JDBC URL. |
| `username(String)` | `username` | – | Database username. |
| `password(String)` | `password` | – | Database password. |
| `readOnlyUrl(String)` | `readOnlyUrl` | – | Optional separate URL for read-only connections. |
| `driver(...)` / `driver(String)` | `driver` (or `databaseDriver`) | auto from URL | JDBC driver class / instance. |
| `schema(String)` | `schema` | driver default | Default schema applied to connections. |
| `catalog(String)` | `catalog` | driver default | Default catalog applied to connections. |
| `isolationLevel(int)` | `isolationLevel` | `READ_COMMITTED` | Transaction isolation level. Property accepts names e.g. `READ_COMMITTED`. |
| `autoCommit(boolean)` | `autoCommit` | `false` | Auto-commit mode for pooled connections. |
| `readOnly(boolean)` | `readOnly` | `false` | Mark connections read-only (optimises read workloads). |
| `applicationName(String)` | `applicationName` | – | Application name reported to the driver where supported. |
| `clientInfo(Properties)` | `clientInfo` | – | Client info properties (semicolon separated `key=value` in properties form). |
| `customProperties(Map)` / `addProperty(...)` | `customProperties` | – | Extra JDBC driver connection properties (semicolon separated `key=value` in properties form). |
| `initSql(List<String>)` | `initSql` | – | SQL run on each new connection (semicolon separated statements in properties form). See per-connection init below. |

## Pool sizing

| Builder method | Property key | Default | Description |
|----------------|--------------|---------|-------------|
| `minConnections(int)` | `minConnections` | `2` | Minimum connections maintained in the pool. |
| `initialConnections(int)` | `initialConnections` | = `minConnections` | Connections created on startup. Set higher than min for smooth warm-up (Kubernetes). |
| `maxConnections(int)` | `maxConnections` | `200` | Maximum connections. Threads block (up to `waitTimeout`) when this is reached. |

## Timeouts, trimming and ageing

| Builder method | Property key | Default | Description |
|----------------|--------------|---------|-------------|
| `waitTimeoutMillis(int)` | `waitTimeout` | `1000` | Millis a thread waits for a free connection once the pool is at max before throwing `ConnectionPoolExhaustedException`. |
| `maxInactiveTimeSecs(int)` | `maxInactiveTimeSecs` | `300` | Idle seconds after which a free connection can be trimmed back towards `minConnections`. |
| `maxAgeMinutes(int)` | `maxAgeMinutes` | `0` (unlimited) | Maximum age of a connection before it is trimmed regardless of activity. |
| `trimPoolFreqSecs(int)` | `trimPoolFreqSecs` | `59` | How often the background trim check runs. |

## Health checks / heartbeat

| Builder method | Property key | Default | Description |
|----------------|--------------|---------|-------------|
| `validateOnHeartbeat(boolean)` | `validateOnHeartbeat` | `true` (`false` in AWS Lambda) | Enable the background heartbeat that validates the pool. |
| `heartbeatFreqSecs(int)` | *(builder only)* | `30` | How often the heartbeat runs. |
| `heartbeatTimeoutSeconds(int)` | `heartbeatTimeoutSeconds` | `30` | Query timeout for the heartbeat validation. |
| `heartbeatSql(String)` | `heartbeatSql` | `Connection.isValid()` / platform default | Explicit validation SQL. Rarely needed — see the validation guide. |
| `heartbeatMaxPoolExhaustedCount(int)` | *(builder only)* | `10` | Consecutive heartbeat pool-exhaustion detections before the pool is reset (leak recovery). |

See [Connection Validation Best Practices](connection-validation-best-practices.md) for details.

## Leak detection / diagnostics

| Builder method | Property key | Default | Description |
|----------------|--------------|---------|-------------|
| `leakTimeMinutes(int)` | `leakTimeMinutes` | `30` | A busy (checked-out) connection older than this is treated as a leak and force-closed during a pool reset. |
| `captureStackTrace(boolean)` | `captureStackTrace` | `false` | Capture the stack trace when a connection is obtained, to locate leaks. Has a performance cost. |
| `maxStackTraceSize(int)` | `maxStackTraceSize` | `5` | Number of stack frames reported for busy connections. |

See [Troubleshooting Connection Leaks & Pool Exhaustion](troubleshooting-connection-leaks.md).

## Statement caching

| Builder method | Property key | Default | Description |
|----------------|--------------|---------|-------------|
| `pstmtCacheSize(int)` | `pstmtCacheSize` | `300` | `PreparedStatement` cache size, per connection. |
| `cstmtCacheSize(int)` | `cstmtCacheSize` | `20` | `CallableStatement` cache size, per connection. |

## Lifecycle / startup

| Builder method | Property key | Default | Description |
|----------------|--------------|---------|-------------|
| `failOnStart(boolean)` | `failOnStart` | `true` | When `false`, the pool starts even if the database is unavailable (it recovers later via heartbeat). |
| `offline(boolean)` | `offline` | `false` | Start the pool offline (no connections created until `online()`). |
| `shutdownOnJvmExit(boolean)` | `shutdownOnJvmExit` | `false` | Register a JVM shutdown hook to close the pool on exit. |
| `enforceCleanClose(boolean)` | `enforceCleanClose` | `false` | Throw if a dirty (uncommitted) connection is closed. Recommended in tests. See [issue #116](https://github.com/ebean-orm/ebean-datasource/issues/116). |

## Hooks and extension points

| Builder method | Property key | Description |
|----------------|--------------|-------------|
| `connectionInitializer(NewConnectionInitializer)` | *(builder only)* | Hook called when each new connection is created (`preInitialize` / `postInitialize`). |
| `defaultConnectionInitializer(NewConnectionInitializer)` | *(builder only)* | Fallback initializer used only if one is not otherwise set. |
| `listener(DataSourcePoolListener)` | *(builder only)* | Callbacks on borrow (`onAfterBorrowConnection`) and return (`onBeforeReturnConnection`). |
| `poolListener(String)` | `poolListener` | Class name of a `DataSourcePoolListener` to instantiate. |
| `alert(DataSourceAlert)` | *(builder only)* | Callbacks for `dataSourceUp` / `dataSourceDown` (outage alerting). See [Monitoring](monitoring-pool-metrics.md). |

## Per-connection initialization

Use `initSql` for simple statements, or a `NewConnectionInitializer` for programmatic control. This
is the right place to set things like a Postgres `search_path` or a per-session `statement_timeout`.

```java
DataSourcePool pool = DataSourcePool.builder()
.name("mypool")
.url("jdbc:postgresql://localhost:5432/myapp")
.username("app_user")
.password("password")
.initSql(List.of("set search_path to app, public"))
.connectionInitializer(new NewConnectionInitializer() {
@Override
public void postInitialize(Connection connection) {
try (Statement st = connection.createStatement()) {
st.execute("set statement_timeout to '30s'");
} catch (SQLException e) {
throw new IllegalStateException(e);
}
}
})
.build();
```

---

## Deprecated `setXxx` methods

Many settings historically used a `setXxx` name (e.g. `setMinConnections`). These remain for
backwards compatibility but are deprecated — prefer the fluent forms shown above
(`minConnections`, `maxConnections`, `heartbeatFreqSecs`, etc.).

---

## Next Steps

- [Create a DataSource Pool](create-datasource-pool.md)
- [Connection Validation Best Practices](connection-validation-best-practices.md)
- [Troubleshooting Connection Leaks & Pool Exhaustion](troubleshooting-connection-leaks.md)
- [Monitoring Pool Metrics](monitoring-pool-metrics.md)
- Full Javadoc: [io.ebean:ebean-datasource](https://javadoc.io/doc/io.ebean/ebean-datasource)
7 changes: 7 additions & 0 deletions docs/guides/create-datasource-pool.md
Original file line number Diff line number Diff line change
Expand Up @@ -164,6 +164,10 @@ DataSourcePool pool = DataSourcePool.builder()

## Configuration Reference

> For the **complete** list of settings, defaults, and property keys see the
> [Configuration Reference](configuration-reference.md). The table below covers the most common
> settings.

### Common Settings

| Setting | Default | Purpose |
Expand Down Expand Up @@ -217,6 +221,9 @@ DataSourcePool pool = DataSourcePool.builder()

## Next Steps

- Read the [Configuration Reference](configuration-reference.md) for all available settings and property keys
- [Troubleshooting Connection Leaks & Pool Exhaustion](troubleshooting-connection-leaks.md)
- [Monitoring Pool Metrics](monitoring-pool-metrics.md)
- Read the [README](../../README.md) for more information about the connection pool
- Check the [ebean-datasource GitHub repository](https://github.com/ebean-orm/ebean-datasource) for latest updates
- Consult the [DataSourceBuilder API documentation](https://javadoc.io/doc/io.ebean/ebean-datasource) for all available configuration options
167 changes: 167 additions & 0 deletions docs/guides/monitoring-pool-metrics.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,167 @@
# Guide: Monitoring Pool Metrics

## Purpose

This guide explains how to observe the health of a `DataSourcePool` at runtime: reading pool metrics
via `PoolStatus`, checking online/up state, and receiving outage notifications via `DataSourceAlert`.

---

## Reading pool metrics with `PoolStatus`

Call `status(reset)` on the pool to get a point-in-time snapshot:

```java
PoolStatus status = pool.status(false); // false = read without resetting counters

System.out.printf(
"min=%d max=%d free=%d busy=%d waiting=%d highWaterMark=%d hitCount=%d waitCount=%d%n",
status.minSize(), status.maxSize(), status.free(), status.busy(),
status.waiting(), status.highWaterMark(), status.hitCount(), status.waitCount());
```

### Metric meanings

| Metric | Meaning |
|--------|---------|
| `minSize()` | Configured minimum connections. |
| `maxSize()` | Configured maximum connections. |
| `free()` | Connections currently idle and available. |
| `busy()` | Connections currently checked out (in use). |
| `size()` | `free() + busy()` — total connections in the pool. |
| `waiting()` | Threads currently blocked waiting for a connection. |
| `highWaterMark()` | Peak `busy()` value seen since the last reset. |
| `hitCount()` | Number of times a connection was requested from the pool. |
| `waitCount()` | Number of times a thread had to wait (pool was full). |
| `totalAcquireMicros()` | Total time spent acquiring connections (micros). |
| `totalWaitMicros()` | Total time threads spent waiting when the pool was full (micros). |
| `maxAcquireMicros()` | Slowest single acquire (micros). |
| `meanAcquireNanos()` | Mean acquire time (nanos) — typically in the ballpark of ~150 nanos. |

### What to watch for

- **`highWaterMark()` approaching `maxSize()`** — the pool is close to exhaustion; consider raising
`maxConnections` or investigate a leak (see
[Troubleshooting Connection Leaks & Pool Exhaustion](troubleshooting-connection-leaks.md)).
- **Non-zero / rising `waitCount()` and `waiting()`** — threads are blocking on the pool; demand
exceeds capacity.
- **`busy()` not dropping when traffic falls** — a likely connection leak.
- **High `maxAcquireMicros()` / `meanAcquireNanos()`** — acquiring is slow, often a sign of
contention or repeated validation/recreation.

### Resetting counters for periodic sampling

Pass `true` to reset the cumulative counters (`highWaterMark`, `hitCount`, `waitCount`, acquire/wait
times) after reading. This is ideal for emitting metrics on a fixed interval:

```java
// every 60s, e.g. from a scheduled task
PoolStatus status = pool.status(true); // read and reset, so the next window starts clean
meterRegistry.gauge("db.pool.busy", status.busy());
meterRegistry.gauge("db.pool.free", status.free());
meterRegistry.gauge("db.pool.highWaterMark", status.highWaterMark());
meterRegistry.gauge("db.pool.waitCount", status.waitCount());
```

---

## Checking pool state

```java
pool.isOnline(); // true once the pool has been brought online (connections created)
pool.isDataSourceUp(); // false if the pool has detected the database is down
pool.size(); // current number of connections (free + busy)

SQLException reason = pool.dataSourceDownReason(); // non-null when the pool is down
```

- `isOnline()` reflects the lifecycle state (`online()` / `offline()`).
- `isDataSourceUp()` reflects connectivity health detected by the heartbeat.

---

## Outage notifications with `DataSourceAlert`

Register a `DataSourceAlert` to be notified when the pool detects the database has gone down and when
it recovers. This is useful for wiring pool health into your alerting/paging system.

```java
DataSourceAlert alert = new DataSourceAlert() {
@Override
public void dataSourceDown(DataSource dataSource, SQLException reason) {
// e.g. send to your alerting system / increment a metric
log.error("Database pool is DOWN", reason);
}

@Override
public void dataSourceUp(DataSource dataSource) {
log.warn("Database pool is back UP");
}
};

DataSourcePool pool = DataSourcePool.builder()
.name("mypool")
// ... url/username/password ...
.alert(alert)
.build();
```

The pool also logs these transitions itself:

```
FATAL: DataSource [mypool] is down or has network error!!!
RESOLVED FATAL: DataSource [mypool] is back up!
```

---

## Borrow/return hooks with `DataSourcePoolListener`

For per-connection observation (e.g. timing, tracing, auditing), register a listener:

```java
DataSourcePoolListener listener = new DataSourcePoolListener() {
@Override
public void onAfterBorrowConnection(Connection connection) {
// called when a connection is handed to the application
}

@Override
public void onBeforeReturnConnection(Connection connection) {
// called just before a connection is returned to the pool
}
};

DataSourcePool pool = DataSourcePool.builder()
.name("mypool")
// ...
.listener(listener)
.build();
```

> Keep listener callbacks fast — they run on the borrow/return path of every connection.

---

## Putting it together: a simple health check

```java
PoolStatus status = pool.status(false);
boolean healthy =
pool.isDataSourceUp()
&& status.waiting() == 0
&& status.busy() < status.maxSize();

if (!healthy) {
log.warn("DB pool degraded: up={} busy={}/{} waiting={}",
pool.isDataSourceUp(), status.busy(), status.maxSize(), status.waiting());
}
```

---

## Next Steps

- [Troubleshooting Connection Leaks & Pool Exhaustion](troubleshooting-connection-leaks.md)
- [Configuration Reference](configuration-reference.md)
- [Connection Validation Best Practices](connection-validation-best-practices.md)
Loading
Loading