# Omni Bridge End-to-End Tests

## General description

The E2E tests cover an entire workflow involving multiple blockchain components (NEAR, EVM-based chains, Solana) and cross-chain communication. These tests ensure that all parts (smart contracts, scripts, etc.) integrate correctly. The [Snakefiles](https://snakemake.readthedocs.io/en/stable/) in this project orchestrate each step in the workflow, from compiling and deploying contracts on various chains to executing and verifying cross-chain transactions.

## Prerequisites

You will need the following tools installed on your environment before proceeding:
- **Snakemake**. Refer to the [Snakemake installation guide](https://snakemake.readthedocs.io/en/stable/getting_started/installation.html) for instructions.
- **Yarn**. Used for installing TypeScript dependencies for EVM contracts and scripts.
- **Cargo**. The Rust package manager, required for building Rust-based components.
- **NEAR CLI RS**. A command-line interface for interacting with NEAR protocols.
- **Docker**. Required to build NEAR contracts in consistent environment.
- **Solana CLI and Anchor**. For compiling and deploying Solana programs.
- **Bridge SDK CLI**. Install with:
`cargo install --git https://github.com/Near-One/bridge-sdk-rs/ --rev <actual_revision> bridge-cli`, look for `<actual_revision>` in the `.github/workflows/e2e-test.yml` file.
- **Cargo Near**. Used to build NEAR contracts. Install with:
`cargo install --locked cargo-near`
Enables bridging functionality for various blockchain environments.

## Project Structure

### Main Directories

- `tools/` - TypeScript-based utilities and scripts that support the E2E testing process:
  - `src/lib/` - Contains shared libraries and utilities used across different scripts
  - `src/scripts/` - Contains the main scripts for deployment, testing, and chain interaction
  - `src/E2ETestToken/` - Contains token-related implementations and utilities

- `snakefiles/` - Contains modular Snakefiles split by functionality:
  - `near.smk`, `evm.smk`, `solana.smk` - Chain-specific build and deployment rules
  - `common.smk` - Shared rules.
  - `<pipeline_number>_<pipeline_name>.smk` - Complex multi-chain testing scenarios (e.g., `01_bridge_token_near_to_evm.smk`)
  - `utils.py` - Contains utility functions used in the Snakefiles.
  - `const.py` - Contains constants used in the Snakefiles. It should be trated as a config to be changed if required.

- `bin/` - Contains compiled binaries and executables used by the testing process.

During the execution of the tests, the `generated/` directory will be created (could be configured in `const.py`). All generated files and artifacts are stored here, including:
- Build outputs
- Deployment results
- Test artifacts
- Transaction receipts
- Contract addresses

### Configuration Files
- `snakefiles/const.py` - Contains constants used in the Snakefiles. It should be trated as a config to be changed if required.
- `bridge-sdk-config.example.json` - Template for bridge SDK configuration
- `near_init_params.json` - NEAR contract initialization parameters
- `tools/.env.example` - Example environment variables file for the tools
- `../evm/.env.example` - Example environment variables file for the EVM contracts
- Various keypair files for Solana program deployment
- `environment.yml` - Conda environment file to set up the environment for Snakemake during the Github Actions workflow (no need to change).

Currently, the values of the config files are sometimes intersected. TODO: refactor to create a single config file for end-to-end tests.

### Github Actions

The repository contains a Github Actions workflow for running the end-to-end tests. The workflow is defined in `.github/workflows/e2e-test.yml`.

The custom action to setup the environment is defined in `.github/e2e-setup/action.yml`.

### Workflows Visualizations

All workflows are visualized into DAGs and stored in `workflows_viz` directory as PDF files.

The files can be regenerated by running the following command:
```
snakemake -s <pipeline_name>.smk --rerun-triggers=mtime --dag <rule_name> | dot -Tpdf > workflows_viz/<pipeline_name>.pdf
```

Here `dot` is a tool from `graphviz` package.

## User guide

Before running the tests, make sure that you have all the prerequisites installed and configured.


### How to run the tests (Github Actions)

**Prerequisites**:
- [The step should be done once by the repository admin] Set up the environment variables used in `e2e-test.yml` file in the repository settings. Currently, this includes `E2E_INFURA_API_KEY`, `E2E_EVM_PRIVATE_KEY` and `E2E_ETH_RPC_URL` (this one should be added without the trailing slash).


Currently, the tests should be triggered manually. In order to do this, you need:
1. Go to the "Actions" tab in the repository.
2. Select "E2E Tests" workflow.
3. Click on "Run workflow" button.
4. Enter the names of the pipelines you want to run, separated by spaces. The name of the pipeline is the stem of the corresponding Snakefile.
5. Enter the tag of the contracts to download. If you want contracts to be built from the source leave it empty.
6. Click on "Run workflow" button.

After the workflow is finished, you can find all the generated files in archive in the Artifacts section of the workflow run page.

### Configuration (locally)

You need to create a `.env` files from the examples provided in:
- `./tools/.env.example`
- `../evm/.env.example` (`INFURA_API_KEY` and `EVM_PRIVATE_KEY` only)

Also you need to copy or rename the provided `bridge-sdk-config.example.json` to `bridge-sdk-config.json`. And update it with your `ETH_PRIVATE_KEY` and your `ETH_RPC` endpoint.

For Solana bulding and deployment, ensure that for every program you have a keypair in `.e2e-testing/` directory in the format of `<program_name>-keypair.json`. However, this key pair is secret and should not be shared.

### How to run the tests (locally)

This repository contains multiple Snakefiles, each focusing on a particular chain or pipeline. Each Snakefile is an independent set of rules that perform a corresponding workflow.

To run a pipeline, you need to execute the following command:
```
snakemake -s snakefiles/<pipeline_name>.smk -j1 --rerun-triggers=mtime
```

### Result artifacts (locally)

Throughout the pipelines, you will see JSON files containing addresses, transaction hashes, and other relevant data.
These files serve as evidence that each step or deployment was successfully executed. They are automatically generated and stored in dedicated directories such as `evm_deploy_results`, `near_deploy_results` etc.


### Handling Pipeline Failures (locally)

If a pipeline fails at a certain step, fix the underlying issue and rerun the same command. Snakemake will pick up from the point of failure if the previous steps have created their artifact files or “stamp” files. Don't forget to add `--rerun-triggers=mtime` to the command to ensure that the pipeline is run from the place it failed.

### Rebuilding parts of pipeline (locally)

In some cases, you might want to rebuild only a part of the pipeline and propagate the changes to the rest of the pipeline if required. E.g. rebuild and redeploy the contract A and everything that depends on it. Please, see the following sections on how to do this.

In the common case (especially with `--rerun-triggers=mtime` being used), Snakemake will detect the changes in the files and will rebuild the targets that depend on the changed files. However, due to the Snakemake implementation specifics, it doesn't detect the deletion of the files. Therefore, let's imagine that you want to rerun the rule `A` and it corresponds to the file `generated/some_pipleine/A.json`. In this case, there could be two cases:
1. It's possible to rerun the rule `A` **without rerunning its prerequisites**. Example: you want to redeploy the contract A and the contract doesn't depend on anything else.
2. It's **required to rerun the prerequisites** of the rule `A` before rebuilding it. Example: the rule sends tokens from contract A to contract B but after the pipeline execution the contract A doesn't have any tokens left. Therefore, you need to rerun the prerequisite rules, i.e. mint (or send) tokens to contract A.

In the **first case** you should do the following:
1. Rerun the desired target: `snakemake -s snakefiles/<pipeline_name>.smk -j1 --rerun-triggers=mtime <target_name>`
2. Rerun the whole pipeline to propagate the changes: `snakemake -s snakefiles/<pipeline_name>.smk -j1 --rerun-triggers=mtime`

In the **second case** you should do the following:
1. Rerunt the desired target and all of its prerequisite rules: `snakemake -s snakefiles/<pipeline_name>.smk -j1 --rerun-triggers=mtime --forceall <target_name>`
2. Rerun the whole pipeline to propagate the changes: `snakemake -s snakefiles/<pipeline_name>.smk -j1 --rerun-triggers=mtime`

If you want to just **rerun the whole pipeline**, you should:
1. Delete all the related files: `snakemake -s snakefiles/<pipeline_name>.smk --delete-all-output`
2. Rerun the pipeline: `snakemake -s snakefiles/<pipeline_name>.smk -j1 --rerun-triggers=mtime`

#### Note on binaries

In contrast to other targets, binaries are not automatically rebuilt when the corresponding source code changes. Therefore, if you want to rebuild binaries, you must delete them or the corresponding stamp files.

This can be done either manually or by running the following command (preferred):
```
snakemake -s snakefiles/<pipeline_name>.smk --delete-all-output
```

The command above will delete all the files which are generated during the pipeline execution. If you want to delete only the files related to some specific rule, you can specify it as follows:
```
snakemake -s snakefiles/<pipeline_name>.smk --delete-all-output <rule_name>
```

## Developer guide

### Introduction to Snakemake

- If you are new to Snakemake, the [Snakemake documentation](https://snakemake.readthedocs.io/en/stable/) is an excellent place to start.
- Snakemake allows us to define rules that specify how to build or process files and manage dependencies in a declarative way.
- Comparing to Makefile, Snakemake has friendlier syntax and more powerful features such as DAGs visualization, effective parallel execution, modularization, built-in message printing and parameters definition.
- In order to have a better experience writing Snakemake, it's recommended to use Snakemake extension for IDE. E.g. [extension for VSCode](https://marketplace.visualstudio.com/items?itemName=Snakemake.snakemake-lang).

### Structure of the Snakefiles

- Each Snakefile is an independent set of rules that perform a corresponding workflow.
- Rules from one Snakefile are reused in another one by using [modules](https://snakemake.readthedocs.io/en/stable/snakefiles/modularization.html#modules).
- In order to avoid name conflicts, rules are included into modules and are prefixed with the module name.
- Since Snakemake can use Python functions and variables, some of them are abstracted into `const.py` and `utils.py` files.

### Debugging

- Use `-n` flag to print the jobs that will be executed without actually running them. Additionally, Snakefile will check if there're any errors which could be detected before the actual execution.
- DAG visualization can also be helpful to understand the dependencies between the rules (see command above).

## Further improvements

The following improvements are nice to be done in the future. They are presented in the subjective order of priority.

- **Unified configuration**. Currently, the values of the config files are scattered and even intersected in different files (see [Configuration Files](#configuration-files) section). It should be refactored to have a single config file for end-to-end tests.
- **Transparent variables reusage**. If we want that some rule A is triggered by rule B, we need to ensure that the path in the `output` of rule A matches the path in the `input` of rule B. 
Currently, we simply define variables with the same paths in different Snakefiles. It's error-prone and should be refactored. Since variables can't be directled imported between Snakefiles, one possible solution is to create a centralized **Python** file(s) (like `const.py` or `utils.py`) and move all paths variables there.
- **Docker containerization**. Put all the dependencies and their installation into a Docker container and run the tests in a consistent environment.
- **Parallel execution of the pipelines**. Even though the dependency graph and Snakemake itself are ready for parallel execution of the tasks, currently, the pipelines are executed sequentially. The reason for this is that parallel execution causes race conditions when the same account tries to submit transactions with the same nonce. However, be aware that this improvement will speed up the execution of the pipelines only locally, since our Github Actions runners are single-threaded at the moment.
- **Reuse contracts/accounts from previous GA runs**. Currently, the contracts are deployed from scratch. However, sometimes it may be useful and cost-effective to reuse the contracts/accounts from previous GA runs. To achieve this, we need to store the desired deployment results and then put them in a corresponding location in the next run.