Plugin Local Development & Test Setup

Goal

Set up local development and test environments for the Keymate gateway authorization plugins. This guide covers two workflows: a standalone local environment for rapid development with a mock Access Gateway, and a full integration test stack with Keycloak, Access Gateway, and audit services.

Audience

Developers working on the APISIX or Istio access plugins who need a local environment for development, testing, and debugging.

Prerequisites

• A container runtime installed locally
• For WASM plugin development: TinyGo 0.33.0 (or use the containerized build)
• For k8s-connected development: kubectl and a running Kubernetes cluster
• For Istio plugin k8s development: Istio installed with sidecar injection
• For performance testing: k6 load testing tool
• Python 3 with the requests and python-dotenv libraries (for integration tests)

Before You Start

Each plugin repository provides two local development workflows under the local/ directory:

Workflow	Directory	Requires Kubernetes	Gateway	Best For
Standalone (Container)	local/docker/	No	Mock (always 200)	Rapid iteration, no external dependencies
k8s-connected	local/containerd/	Yes	Real Access Gateway in cluster	End-to-end authorization testing

Additionally, the APISIX plugin includes a full integration test stack under test/docker/ that starts all services (Keycloak, Access Gateway, Audit Collector) for complete authorization flow testing.

Worked Example

In this guide, you set up the standalone development environment, run a smoke test, then optionally start the full integration stack and run the Python test suite.

Steps

1. Standalone development (APISIX plugin)

Start a lightweight local stack with Apache APISIX, an echo server, and a mock Access Gateway that always returns 200:

Start standalone APISIX environment

cd local/docker
make up

This starts three containers:

Service	Purpose	Local Port
Apache APISIX 3.10.0	APISIX with the Lua plugin mounted	9080
Echo Server	Test upstream service	—
Mock Gateway	Simulates Access Gateway (always returns PERMIT)	—

Run the k6 smoke test to verify the setup:

Run k6 smoke test

make test

The smoke test validates that:

• GET requests return 200 with Keymate-Gateway-Auth-Duration and Keymate-Gateway-Auth-Status: GRANTED headers
• POST requests with a body return 200

View APISIX logs and stop the stack:

View logs and stop

make logs
make down

2. Standalone development (Istio plugin)

The Istio plugin standalone environment uses an Envoy proxy directly instead of APISIX:

Start standalone Istio plugin environment

cd local/docker
make up

Service	Purpose	Local Port
Envoy Proxy	Envoy with the Lua filter loaded	10000
Echo Server	Test upstream service	—
Mock Gateway	Simulates Access Gateway (always returns PERMIT)	—

Run the smoke test:

Run k6 smoke test

make test

The Istio plugin smoke test additionally validates that requests to /health receive Keymate-Gateway-Auth-Status: SKIPPED (skip path behavior).

note

When editing the Istio Lua plugin locally, update both lua/keymate-access.lua (production config) and local/config/keymate-access-local.lua (local dev config with mock gateway cluster settings).

3. Build the WASM plugin

Both plugins compile WASM binaries from Go source using TinyGo inside a container:

Build WASM binary

make build

This runs tinygo build -o plugin.wasm -scheduler=none -target=wasi -gc=conservative inside a TinyGo 0.33.0 container and produces the compiled binary at wasm/plugin.wasm.

To build the full container image:

Build plugin container image

make image

For the APISIX plugin, the container image packages both the WASM binary (/plugins/wasm/keymate-access-gateway.wasm) and the Lua plugin (/plugins/lua/keymate-access-gateway.lua). For the Istio plugin, the image contains only the WASM binary (/plugin.wasm).

note

The Lua plugins do not require a compilation step. Edit the Lua source file directly and reload the gateway configuration to apply changes.

4. k8s-connected development

For testing against a real Access Gateway running in your Kubernetes cluster:

Build, deploy, and test in k8s

cd local/containerd
make image    # Build WASM + Lua, create image, load to k8s.io namespace
make deploy   # Restart the gateway pod to pick up the new plugin image
make test BASE_URL=https://api-local.example.com
make logs     # Tail gateway logs

Available make targets:

Target	Description
build	Compile WASM plugin with TinyGo
image	Build container image and load to k8s.io namespace
deploy	image + restart the gateway pod in Kubernetes
test	Run k6 smoke test (BASE_URL configurable)
logs	Tail gateway proxy logs via kubectl
clean	Remove WASM binary

Override defaults:

Deploy with custom settings

make deploy IMAGE_TAG=registry.example.com/keymate-access-gateway:v1.0.0 NAMESPACE=my-namespace

5. Start the full integration test stack (APISIX)

The APISIX plugin includes a full integration test stack under test/docker/ that runs all required services:

Service	Purpose	Local Port
Apache APISIX 3.10.0	APISIX with the plugin mounted	9080
Access Gateway	Authorization decision service	8282
Keycloak	Identity provider for token generation	8180
Relational Database 15	Database backend for Keycloak	5435
Infinispan 15.0	Cache backend	11222
Audit Collector	Audit logging service	8097
HTTPBin	Test upstream service	3000

Start the full integration stack

cd test/docker
make up

Wait for all services to become healthy. Keycloak and the Access Gateway take 1–2 minutes to initialize.

6. Obtain a test token

Request an access token from Keycloak:

Get access token from Keycloak

TOKEN=$(curl -s -X POST http://localhost:8180/realms/<REALM>/protocol/openid-connect/token \
  -d "grant_type=client_credentials" \
  -d "client_id=<CLIENT_ID>" \
  -d "client_secret=<CLIENT_SECRET>" \
  | python3 -c "import sys,json; print(json.load(sys.stdin)['access_token'])")

7. Test a request through the plugin

Send a request through a gateway route configured with the plugin:

Test the plugin

curl -v http://localhost:9080/<ROUTE_PATH> \
  -H "Authorization: Bearer $TOKEN" \
  -H "Keymate-Client-Id: <CLIENT_ID>"

A successful authorization returns a 200 response with decision headers:

< Keymate-Decision: ...
< Keymate-Decision-Latency: 45
< Keymate-Decision-Cache: miss
< Keymate-Gateway-Auth-Duration: 52.350
< Keymate-Gateway-Auth-Status: GRANTED

A denied request returns the Access Gateway's error response with the corresponding HTTP status code and Keymate-Gateway-Auth-Status: DENIED.

8. Run integration tests

The repository includes a Python integration test suite:

Run integration tests

python3 test_apisix_plugin.py

The test suite:

1. Authenticates with Keycloak
2. Sends a GET request expected to succeed (HTTP 200) through the Lua route
3. Sends a DELETE request expected to be denied (HTTP 403) through the Lua route
4. Repeats the same tests through the WASM route
5. Tests the direct Access Gateway endpoint for comparison

Test results are logged to test_apisix_plugin.log. Configure the test via environment variables in .env:

Variable	Default	Description
KEYCLOAK_HOST	auth-local.*.sslip.io	Keycloak hostname
API_HOST	api-local.*.sslip.io	APISIX hostname
PROTOCOL	https	HTTP or HTTPS
CLIENT_ID	—	Keycloak client ID
CLIENT_SECRET	—	Keycloak client secret
USERNAME	—	Test user username
PASSWORD	—	Test user password

9. Run performance tests (optional)

Use k6 for load testing the plugin:

Run k6 performance tests

k6 run \
  -e PROTOCOL=https \
  -e API_HOST=api-dev.example.com \
  -e KEYCLOAK_HOST=auth-dev.example.com \
  -e CLIENT_ID=<CLIENT_ID> \
  -e CLIENT_SECRET=<CLIENT_SECRET> \
  -e USERNAME=<USERNAME> \
  -e PASSWORD=<PASSWORD> \
  k6-performance-test.js

The performance test authenticates with Keycloak during setup and measures plugin and gateway latencies.

10. Development workflow

For Lua changes:

1. Edit the Lua source file (lua/keymate-access-gateway.lua for APISIX, lua/keymate-access.lua for Istio)
2. Reload the gateway configuration or restart the gateway container
3. Test with cURL, k6, or the Python integration test suite

For WASM changes:

1. Edit wasm/main.go
2. Build: make build
3. Rebuild the container image: make image
4. Restart the gateway container to load the new WASM binary
5. Test with cURL, k6, or the Python integration test suite

11. Clean up

Stop the integration test stack:

Stop the test environment

cd test/docker
make down

Validation Scenario

Scenario

You start the standalone container environment, run the k6 smoke test, and confirm the plugin headers appear in the response.

Expected Result

1. All containers start and respond to requests
2. The k6 smoke test passes with Keymate-Gateway-Auth-Status: GRANTED
3. Response headers include Keymate-Gateway-Auth-Duration

How to Verify

• API evidence: The cURL response shows HTTP 200 with upstream response body and decision headers
• Logs: Gateway logs show plugin execution entries
• Audit evidence: When using the full integration stack, the Audit Collector service receives the authorization event

Troubleshooting

• Services fail to start — Ensure the container runtime has enough memory allocated (minimum 4 GB recommended). Check service logs for specific errors.
• Keycloak not ready — Keycloak can take 60–120 seconds to initialize. Wait for the health check to pass before requesting tokens.
• make build fails — The build requires pulling the TinyGo image. Ensure the container runtime is running and has internet access.
• Token request fails — Verify the Keycloak realm, client ID, and client secret match the test configuration. Check that Keycloak is reachable at localhost:8180.
• Plugin returns 502 — The Access Gateway may not be ready yet. Check the Access Gateway service logs and wait for the health check to pass.
• WASM changes not taking effect — After make build, restart the gateway container to load the new WASM binary. WASM modules are loaded at startup.
• Mock gateway returns unexpected responses — The standalone mock gateway always returns HTTP 200. For authorization testing with real decisions, use the full integration stack or the k8s-connected workflow.

Next Steps

To understand the plugin architecture, configuration options, and extension points, see Extending the APISIX Plugin or Extending the Istio Plugin.

Related Docs

Extending the APISIX Plugin

Plugin architecture, configuration, and customization points.

Extending the Istio Plugin

Istio service mesh plugin configuration, deployment, and extension points.

Access Gateway Overview

Endpoints, headers, and system role of the Access Gateway.

Contributing to Enforcement Plugins

Contribution workflow, code standards, and release process.