Skip to content

Dev

Development

Metaport is built with PHP using Silverstripe Framework and is designed and built to run with Docker.

Contributing

First up, have a squiz at how to contribute.

We make heavy use of Continous Integration (CI). Our workflow looks something like this (but it does change sometimes!):

  • Push When a branch is pushed, some basic linting tasks are run. They can also be run locally, which increases the chance of these passing first time. Look in .ci/jobs/lint.
  • Merge Request When a merge request is received, almost everything is run: Lint, Test and Deploy (Don't worry about "deploy", this just refers to auto-generated and deployed docs).
  • Tag When a new GIT tag is received, the audit job is run, a new image is produced and pushed to our Docker Hub, with an accompanying release created in Gitlab.

Requirements

  • A recent version of Docker which comes with the compose command e.g. >= v19.03.0.
  • If making CSS changes to the "metaport-ui" meta-theme: npm >= v18.

Procedure

  • Clone the contents of the repository:
git clone https://gitlab.com/dcentrica/metaport/metaport-server.git && cd metaport-server
  • Configure a local .env file. You can review the contents of .env.example then rename it to .env to suit your own setup.
  • Configure your workstation/laptop/PC to listen for requests to metaport.dev by modifying your system's hosts file:
127.0.0.1   metaport.dev
  • Bring up all the Docker services with docker compose:
docker compose -f docker-compose-dev.yml up -d --build
  • (Optional) If you need the included SSO services (OpenLDAP and Keycloak), then use this command instead:
docker compose -f docker-compose-dev.yml -f docker-compose-idp.yml up -d --build
  • Import all the component end-of-life data:
docker compose -f docker-compose-dev.yml exec app ./vendor/bin/sake dev/tasks/componentimport
  • Create the master encryption key:
docker compose -f docker-compose-dev.yml exec app bash -c 'chmod +x ./bin/setup.sh && ./bin/setupy.sh'
  • Optionally run the entire unit test suite:

Grant privileges over the test database:

sudo docker compose -f docker-compose-dev.yml exec db mysql -uroot -padmin mysql -e "GRANT ALL ON \`ss\_tmpdb\_%\`.* TO \`runner\`@\`%\` IDENTIFIED BY 'metaport'"

Execute the test suite:

docker compose -f docker-compose-dev.yml exec app ./vendor/bin/phpunit
  • Point your browser at https://metaport.dev. (You can safely ignore unsigned certificate warnings from your browser).
  • Now follow the Quickstart Guide.
  • Later, run the following to connect to your team's mailbox (for apps reporting-in using Email transport).
docker compose -f docker-compose-dev.yml exec app ./vendor/bin/sake dev/tasks/mailboxfetch
  • If making CSS changes, the build command is: cd metaport-ui/client/src && nvm use && npm i && vite build

Sending Data

Install one of the available agents into your app, and follow the instructions. For evaluation purposes, there's a basic Laravel app found on our Gitlab which you can test with.

Extending

Metaport is built with the Silverstripe Framework for PHP which has its own module ecosystem. If you're familliar with Silverstripe, then ensure you follow the same design patterns.

Adding New Schedules

Adding new Policy schedules is easy: Copy+Paste an existing one and modify its getSchedule() and readable_schedule() methods as applicable. Use CronMaker for a GUI-based cron syntax generator.

Add new EOL Backend

By default, Metaport uses the API provided by endoflife.date, but alternative backends can be used instead. The following instructions are relevant to Metaport CE in self-managed setups.

Warning

It is not recommended to mix data from multiple backends. Doing so will likely cause confusion at best, and data-corruption at the worst.

  • Create a new directory under plugins.
  • Write a new PHP class e.g. MyMetaportEOLBackend.php which implements BackendProvider.
  • Ensure the environment variable MP_COMPONENT_MANAGER is set correctly e.g. MP_COMPONENT_MANAGER=MyMetaportEOLBackend
  • Refresh Metaport's knowledge of the new class: ./vendor/bin/sake dev/build flush=1
  • Manually run the import task: ./vendor/bin/sake dev/tasks/componentimport (This should already be setup to run automatically via cron)

Add new Dependency Manager Backend

When the MP_DM_BACKEND environment variable is set, Metaport uses the "Classic" dependency and security vulnerability backend. However, it can also talk to the API exposed to it from a local instance of DependencyTrack, and alternative backends can be used as well - though they'll need to be built.

Note

It is OK to run a different Dependency Manager after a previous one, but the connection details will obviously need to be updated.

  • Create a new directory under plugins.
  • Write a new PHP class e.g. MyMetaportDependencyBackend.php which implements BackendProvider.
  • Ensure the environment variable MP_COMPONENT_MANAGER is set correctly e.g. MP_DEPMANAGER=MyMetaportDependencyBackend
  • Refresh Metaport's knowledge of the new class: ./vendor/bin/sake dev/build flush=1
  • Login to Metaport as an admin user, and add the connection details to the relevant team UI (If you're running with multiple teams, this process will need to be repeated for each).

SSO

This project comes with docker compose service config for OpenDLAP and phpLDAPadmin in order to work with the Keycloak IDP. These are only required if you're developing with SSO in mind or doing anything with Keycloak.

To build the necessary services, ensure .env is populated as appropriate (see .env.example) and then run the following, otherwise, Metaport will work fine without SSO using its own authentication system:

docker compose -f docker-compose.yml -f docker-compose-idp.yml up -d

phpLDAPadmin

phpLDAPadmin is just used as a frontend GUI to OpenLDAP which doesn't have its own UI. If you've built your environment with the SSO services, then you should be able to navigate to http://localhost:8082 and login with the following credentials:

  • Username: cn=admin,dc=my-company,dc=com
  • Password: admin

This project comes with a basic hydration export: .dev/config/phpldapadmin-export.ldif. Select the "Import" icon on the top-left and follow the instructions (and ignore the errors).

Keycloak

Review the KEYCLOAK_XX environment variables as documented in .env.example and transpose into your own .env file or wherever you manage your environment variables.

Note

Using the docker compose service name keycloak as the value for the KEYCLOAK_URL environment variable won't work for curl. Similarly, we cannot use localhost as that's meaningless in a Docker context. What we need to do is to use the IP assigned to your PC/Laptop/Workstation which has the same effect.

Tip

Use this command to get the appropriate IP (assumes basic home router using DHCP) and set that to the value of the KEYCLOAK_URL environment variable:

ifconfig | grep 192 | awk '{print $2}'
192.168.20.71

To configure Keycloak itself, visit http://localhost:18080/admin and login with the following credentials:

  • Username: admin
  • Password: admin

Tip

To quickly hydrate the LDAP server, import the provided LDIF export: .dev/config/phpldapadmin-export.ldif.

Tip

To setup a Keycloak client quickly, use the provided export: .dev/config/keycloak-client-export.json. (You'll still need to manually setup an LDAP provider using the guide below).

Tip

Review the comments in app/_config/sso.yml to guide your Keycloak setup.

Connection and authentication settings

Keycloak > User federation > Add Ldap Providers

UI display name: OpenLDAP (Or whatever you like)
Vendor: Other
Connection URL: ldap://ldap-host (Dev only, substitute for your **actual** LDAP host)

Bind type: Simple
Bind DN: cn=admin,dc=my-company,dc=com (Dev only, substitute for your **actual** LDAP bind DN)
Bind credentials: admin (Dev only, substitute for your **actual** LDAP bind password)
LDAP searching and updating
Edit mode: WRITEABLE
Users DN: ou=People,dc=my-company,dc=com (Dev only, substitute for your **actual** LDAP users DN)

Leave everything else as per the defaults.

Dependency Managers

Metaport is capable of supporting multiple Dependency Manager backends. Out of the box there's support for the following backends:

Classic

Classic simply uses the default facilities of the agent's dependency manager e.g. for Composer we invoke the audit and show sub-commands and send that data over the wire to Metaport.

Classic mode will only report dependency and security vulnerability information about application/framework dependencies, and not the O/S or language runtime. If your organisation has access to standalone dependency management software such as Dependency Track, then that will provide everything.

To engage classic mode, both the server and agent need to be told using the MP_DEPMANAGER=Classic environment variable and --classic=1 parameter respectively. See the agent docs for more detailed information.

The Dependency Track backend requires the use of docker-copose-dt.yml.

Self-Signed Certificates

At time of writing, Dependency Track's Docker setup does not run with an HTTPS-enabled configuration. The following command is used to generate a self-signed certificate. Once complete, run docker-compose -f docker-compose-dt.yml up -d to bring it up, add 127.0.0.1 to /etc/hosts and then visit: https://dependencytrack.dev:8443/ in a browser.

mkdir -p .dev/services/dt &&
openssl req \
  -subj "/C=GB/CN=dependencytrack.dev" \
  -addext "subjectAltName = DNS:dependencytrack.dev" \
  -x509 \
  -nodes \
  -days 365 \
  -newkey rsa:2048 \
  -keyout .dev/services/dt/self-signed.key \
  -out .dev/services/dt/self-signed.crt

Using Dependency Track

  1. Get info about a project:
curl -k "https://dependencytrack.dev:8443/api/v1/bom/cyclonedx/project/$DT_PROJECT_UUID" -H 'Content-Type: application/json' -H "X-API-Key: $DT_TEAM_API_KEY"
  1. Generate an SBOM (PHP via Composer)
composer CycloneDX:make-sbom --output-format=JSON --output-file="sbom-composer.json"
  1. Encode
echo "{\"project\": \"$DT_PROJECT_UUID\",\"bom\": \"$( cat ./sbom-composer.json | base64 -w 0 - )\"}" > sbom-composer.encoded.json
  1. Submit
curl -k --fail -X "PUT" "https://dependencytrack.dev:8443/api/v1/bom" -H 'Content-Type: application/json' -H "X-API-Key: $DT_TEAM_API_KEY" -d @sbom-composer.encoded.json

Configure Metaport

  1. Navigate to your team's "settings" tab
  2. Expand the "Dependency Manager Settings" control (If you don't see this, refer to the troubleshooting guide
  3. The Host field is the name of the Docker service & port e.g. http://dtrack-apiserver:8080
  4. The API Token field is the value of the team API token which Dependency Track generates for you when creating a team
  5. The Team Identifier is the UUID of your team in Dependency Track (it can be found using a call to the /v1/api/team endpoint)
  6. Navigate to the desired Metaport application and in its "settings" tab add the Dependency Track project UUID to the Dependency Manager Project ID field. The project UUID can be found from Dependency Track's UI.

Logging

Limited error context is available in the system log /var/log/metaport.log.