# How to Set Up and Run the Max User Benchmark Test

## Purpose

This guide explains how to set up and run a JMeter database benchmark test using PowerShell. The test automatically increases the number of concurrent users until it finds the maximum capacity of your database.

## What This Test Does

The script runs multiple JMeter tests, starting with 50 users and increasing by 25 users each time (50, 75, 100, 125, etc.) up to a maximum of 400 users. Each test runs for 2 minutes. The test stops early if:

- Error rate exceeds 5%
- Response time (P95) exceeds 10 seconds
- Connection pool errors occur

## Prerequisites - Step by Step

1. **Download the latest pipeline build** from [GitLab](https://gitlab.com/scheinone/exan/axium/postgresql-migration)
   - Connect your GitLab account

   ![GitLab Connection](image-20260204-194256.png)

2. **Extract the folder**. You should see this:

   ![Extracted Folder](image-20260204-194330.png)

## Running the Test

1. In the extracted folder, navigate to the test folder and **open PowerShell**

2. Navigate to the config file: `..\hs1-jmeter\config\appsettings.json`

3. **Enter the correct Oracle and Postgres passwords**

   ![Configuration](image-20260204-194445.png)

4. **Run the script**: `hs1-jmeter\run.ps1`

## What Happens During the Test

### Overview

- Check for Java 8 installation
- Start the first test with 50 users
- Display progress every 30 seconds
- After each test completes, analyze results
- If thresholds are not exceeded, increase users by 25 and run again
- Continue until max users (400) is reached or thresholds are exceeded

## What You'll See on Screen

During each test run, you'll see output like:

![Test Output](image-20260128-022419.png)

```
=== Running 50 users ===
WARN StatusConsoleListener The use of package scanning to locate plugins is deprecated and will be removed in a future release
WARN StatusConsoleListener The use of package scanning to locate plugins is deprecated and will be removed in a future release
WARN StatusConsoleListener The use of package scanning to locate plugins is deprecated and will be removed in a future release
WARN StatusConsoleListener The use of package scanning to locate plugins is deprecated and will be removed in a future release
Creating summariser <summary>
Created the tree successfully using .\Postgres-MaxUsers-DBBenchmark.jmx
Starting standalone test @ January 28, 2026 1:58:37 AM UTC (1769565517345)
Waiting for possible Shutdown/StopTestNow/HeapDump/ThreadDump message on port 4446
summary +    392 in 00:00:22 =   17.7/s Avg:   526 Min:   107 Max:  2214 Err:     0 (0.00%) Active: 19 Started: 19 Finished: 0
summary +    595 in 00:00:30 =   19.8/s Avg:  1524 Min:   879 Max:  2167 Err:     0 (0.00%) Active: 44 Started: 44 Finished: 0
summary =    987 in 00:00:52 =   18.9/s Avg:  1127 Min:   107 Max:  2214 Err:     0 (0.00%)
summary +    599 in 00:00:30 =   20.0/s Avg:  2431 Min:  2101 Max:  2637 Err:     0 (0.00%) Active: 50 Started: 50 Finished: 0
summary =   1586 in 00:01:22 =   19.3/s Avg:  1620 Min:   107 Max:  2637 Err:     0 (0.00%)
summary +    611 in 00:00:30 =   20.3/s Avg:  2474 Min:   376 Max:  2656 Err:     0 (0.00%) Active: 50 Started: 50 Finished: 0
summary =   2197 in 00:01:52 =   19.6/s Avg:  1857 Min:   107 Max:  2656 Err:     0 (0.00%)
summary +    206 in 00:00:09 =   22.4/s Avg:  2380 Min:    42 Max:  2604 Err:     0 (0.00%) Active: 0 Started: 1 Finished: 1
summary =   2403 in 00:02:01 =   19.8/s Avg:  1902 Min:    42 Max:  2656 Err:     0 (0.00%)
Tidying up ...    @ January 28, 2026 2:00:39 AM UTC (1769565639229)
... end of run
Users=50  Total=4909  Errors=56  ErrorRate=1.14%  P95=2577ms
=== Running 75 users ===
....
```

After each test completes, you'll see a summary:

```
Users=50  Total=14108  Errors=0  ErrorRate=0.00%  P95=245ms
```

## Understanding the Output

### Summary Lines (during test)

```
summary + 3527 in 00:00:30 = 117.6/s Avg: 173 Min: 78 Max: 413 Err: 0 (0.00%) Active: 33 Started: 33 Finished: 0
```

This means:

- **3527**: Number of requests completed in the last 30 seconds
- **117.6/s**: Throughput (requests per second)
- **Avg: 173**: Average response time in milliseconds
- **Min: 78**: Minimum response time in milliseconds
- **Max: 413**: Maximum response time in milliseconds
- **Err: 0 (0.00%)**: Number of errors and error percentage
- **Active: 33**: Currently active user threads
- **Started: 33**: Total threads started
- **Finished: 0**: Threads that have completed

### Final Summary (after each test)

```
Users=50  Total=14108  Errors=0  ErrorRate=0.00%  P95=245ms
```

This means:

- **Users**: Number of concurrent users tested
- **Total**: Total number of requests completed
- **Errors**: Number of failed requests
- **ErrorRate**: Percentage of requests that failed
- **P95**: 95th percentile response time (95% of requests were faster than this)

## When the Test Stops

The test will stop if:

- Maximum users (400) is reached
- Error rate exceeds 1% (ErrorRateThreshold)
- P95 response time exceeds 10 seconds (P95ThresholdMs)
- Connection pool errors occur

You'll see:

```
STOP: threshold hit at X users
```

![Stop Threshold](image-20260128-022712.png)

## How Long Does It Take

Each individual test runs for:

- **Ramp-up time**: 60 seconds (users gradually start)
- **Test duration**: 120 seconds (2 minutes of actual testing)
- **Total per test**: Approximately 3 minutes

If the test runs from 50 to 400 users (14 steps):

- **Minimum time**: About 42 minutes (if it stops early)
- **Maximum time**: About 42 minutes (if it completes all steps)

The script will show progress, so you can estimate remaining time.

## Results Files

After running, you'll find result files in the `reports\` folder:

![Results Folder 1](image-20260205-222146.png)

![Results Folder 2](image-20260205-222116.png)

These files contain detailed test results that can be analyzed later.

## Modifying Test Settings

To change test behavior, edit `test\benchmark_test.ps1` and modify these variables:

```powershell
$Users = 50          # Starting number of users
$Step = 25           # How many users to add each iteration
$MaxUsers = 400      # Maximum users to test
$JWorkUnits = 1500000 # Rows per query (recursive CTE 1..N)
$P95ThresholdMs = 10000      # Stop if P95 exceeds this (milliseconds)
$ErrorRateThreshold = 0.01   # Stop if error rate exceeds this (1% = 0.01)
```

### Same expensive query on Oracle and Postgres

Both plans use the **same algorithm**: a recursive CTE that generates rows 1..`work_units`, then `SUM(n*n)` and `MAX(n)`. Only the dialect differs (Oracle `WITH t(...)` vs Postgres `WITH RECURSIVE t(...)`). Because the workload is the same, one `$JWorkUnits` value taxes both databases similarly;

The script also uses these JMeter parameters:

- `ramp_seconds=60`: Time to gradually start all users
- `duration_seconds=120`: How long each test runs (2 minutes)


## Benchmark Results

### Oracle

Oracle reached 50% error rate after 75 users

![Oracle Results](image-20260128-023239.png)

### Postgres

Postgres 95th percentile response time was slower than 10 seconds after 225 users

![Postgres Results](image-20260128-023334.png)

## Troubleshooting

### Problem: "Java 8 not found under Program Files\Java"

**Solution:**

- Verify Java 8 is installed: `java -version`
- If installed but not found, ensure it's in `C:\Program Files\Java\` or `C:\Program Files (x86)\Java\`
- Reinstall Java 8 if needed

### Problem: "Java executable not found"

**Solution:**

- Check that Java 8 is properly installed
- The script looks for `java.exe` in the Java installation folder
- Try reinstalling Java 8

### Problem: "Unsupported Java version"

**Solution:**

- You may have Java 9 or higher installed
- Install Java 8 specifically (the script requires Java 8)
- You can have multiple Java versions installed

### Problem: "No results file found, stopping"

**Solution:**

- JMeter may have failed to start
- Check the jmeter log files (`jmeter_oracle_XX.log`) for errors
- Verify the .jmx file exists in the `MaxUserBenchmarkTest` folder
- Check that JMeter path is correct (script looks for `..\bin\jmeter.bat`)

### Problem: "Error preloading the connection pool"

**Solution:**

- Database connection issues
- Verify database URL, username, and password are correct
- Check that database server is accessible
- Verify JDBC driver is in the lib folder
- Check network connectivity to database

### Problem: Script won't run (execution policy error)

**Solution:**

- PowerShell may block script execution
- Run: `Set-ExecutionPolicy -ExecutionPolicy RemoteSigned -Scope CurrentUser`
- Then try running the script again

### Problem: Database connection timeout

**Solution:**

- Verify database URL format is correct
  - For Oracle: `jdbc:oracle:thin:@hostname:port/service`
  - For Postgres: `jdbc:postgresql://hostname:port/database`
- Check firewall settings
- Verify database server is running
