Skip to content
GitLab
Unverified Commit bf9de2d9 authored by mergify[bot]'s avatar mergify[bot] Committed by GitHub
Browse files

Refactors testnetify and remove last localOsmosis Dockerfile (backport #2795) (#2822) 

* Refactors testnetify and remove last localOsmosis Dockerfile (#2795)

* Refactor testnetify and remove Dockerfile

* Manually replace new validator public key

* Improve output formatting

* Bug fixes

* Add ACCOUNT_PUBKEY and ACCOUNT_ADDRESS input

* Add environment variables as input and config folder check

* Update docs, Makefile and rename folder

* Modify CLI arguments

* Update docker-compose.yaml defaults and print updated values

* Invert printing logic replacing verbose with quiet

(cherry picked from commit 3fa5450e

)

# Conflicts:
#	tests/localosmosis/README.md

* fix md conflicts

Co-authored-by: default avatarNiccolo Raspa <6024049+niccoloraspa@users.noreply.github.com>
Co-authored-by: default avatarAdam Tucker <adam@osmosis.team>
parent 28602791
Hide whitespace changes
Inline Side-by-side
Showing
with 521 additions and 459 deletions
+521 -459
Makefile
+ 13
- 7
View file @ bf9de2d9
......@@ -354,26 +354,32 @@ localnet-keys:
localnet-build:
@DOCKER_BUILDKIT=1 docker-compose -f tests/localosmosis/docker-compose.yml build
localnet-build-state-export:
@docker build -t local:osmosis-se --build-arg ID=$(ID) -f tests/localosmosis/mainnet_state/Dockerfile-stateExport .
localnet-start:
@docker-compose -f tests/localosmosis/docker-compose.yml up
localnet-startd:
@docker-compose -f tests/localosmosis/docker-compose.yml up -d
localnet-start-state-export:
@docker-compose -f tests/localosmosis/mainnet_state/docker-compose-state-export.yml up
localnet-stop:
@docker-compose -f tests/localosmosis/docker-compose.yml down
localnet-remove: localnet-stop
rm -rf $(PWD)/tests/localosmosis/.osmosisd
localnet-build-state-export:
@DOCKER_BUILDKIT=1 docker-compose -f tests/localosmosis/state_export/docker-compose.yml build
localnet-start-state-export:
@docker-compose -f tests/localosmosis/state_export/docker-compose.yml up
localnet-startd-state-export:
@docker-compose -f tests/localosmosis/state_export/docker-compose.yml up -d
localnet-stop-state-export:
@docker-compose -f tests/localosmosis/docker-compose.yml down
localnet-remove-state-export:
@docker-compose -f tests/localosmosis/mainnet_state/docker-compose-state-export.yml down
rm -rf $(PWD)/tests/localosmosis/state_export/.osmosisd
.PHONY: all build-linux install format lint \
go-mod-cache draw-deps clean build build-contract-tests-hooks \
......
tests/localosmosis/README.md
+ 37
- 21
View file @ bf9de2d9
......@@ -27,9 +27,12 @@ You can now quickly test your changes to Osmosis with just a few commands:
## LocalOsmosis with Mainnet State
Running LocalOsmosis with mainnet state is resource intensive and can take a bit of time. It is recommended to only use this method if you are testing a new feature that must be thoroughly tested before pushing to production.
Running LocalOsmosis with mainnet state is resource intensive and can take a bit of time.
It is recommended to only use this method if you are testing a new feature that must be thoroughly tested before pushing to production.
A few things to note before getting started. The below method will only work if you are using the same version as mainnet. In other words, if mainnet is on v8.0.0 and you try to do this on a v9.0.0 tag or on main, you will run into an error when initializing the genesis. (yes, it is possible to create a state exported testnet on a upcoming release, but that is out of the scope of this tutorial)
A few things to note before getting started. The below method will only work if you are using the same version as mainnet. In other words,
if mainnet is on v8.0.0 and you try to do this on a v9.0.0 tag or on main, you will run into an error when initializing the genesis.
(yes, it is possible to create a state exported testnet on a upcoming release, but that is out of the scope of this tutorial)
Additionally, this process requires 64GB of RAM. If you do not have 64GB of RAM, you will get an OOM error.
......@@ -52,67 +55,80 @@ systemctl stop osmosisd.service
4. Take a state export snapshot with the following command:
```sh
cd $HOME
osmosisd export 2> testnet_genesis.json
osmosisd export 2> state_export.json
```
After a while (~15 minutes), this will create a file called `testnet_genesis.json` which is a snapshot of the current mainnet state.
After a while (~15 minutes), this will create a file called `state_export.json` which is a snapshot of the current mainnet state.
5. Copy the `state_export.json` to the `localosmosis/state_export` folder within the osmosis repo
5. Copy the `testnet_genesis.json` to the localosmosis folder within the osmosis repo
```sh
cp -r $HOME/testnet_genesis.json $HOME/osmosis/tests/localosmosis
cp $HOME/state_export.json $HOME/osmosis/tests/localosmosis/state_export/
```
6. Ensure you have docker and docker compose installed/running:
Docker
6. Ensure you have docker and docker-compose installed:
```sh
# Docker
sudo apt-get remove docker docker-engine docker.io
sudo apt-get update
sudo apt install docker.io -y
```
Docker Compose
```sh
# Docker compose
sudo apt install docker-compose -y
```
7. Compile the local:osmosis-se docker image (~15 minutes, since this process modifies the testnet genesis you provided above). You may change the exported ID to whatever you want the chain-id to be. In this example, we will use the chain-id of localosmosis.
```sh
cd $HOME/osmosis
export ID=local
7. Build the `local:osmosis` docker image:
```bash
make localnet-build-state-export
```
8. Run the `local:osmosis` docker image
8. Start the local:osmosis-se docker image
```sh
make localnet-start-state-export
```
You will then go through the genesis intialization process. This will take ~15 minutes. You will then hit the first block (not block 1, but the block number after your snapshot was taken), and then you will just see a bunch of p2p error logs with some KV store logs. **This will happen for about 1 hour**, and then you will finally hit blocks at a normal pace.
When running this command for the first time, `local:osmosis` will:
- Modify the provided `state_export.json` to create a new state suitable for a testnet
- Start the chain
You will then go through the genesis initialization process. This will take ~15 minutes.
You will then hit the first block (not block 1, but the block number after your snapshot was taken), and then you will just see a bunch of p2p error logs with some KV store logs.
**This will happen for about 1 hour**, and then you will finally hit blocks at a normal pace.
9. On your host machine, add this specific wallet which holds a large amount of osmo funds
```sh
echo "bottom loan skill merry east cradle onion journey palm apology verb edit desert impose absurd oil bubble sweet glove shallow size build burst effort" | osmosisd keys add wallet --recover --keyring-backend test
MNEMONIC="bottom loan skill merry east cradle onion journey palm apology verb edit desert impose absurd oil bubble sweet glove shallow size build burst effort"
echo $MNEMONIC | osmosisd keys add wallet --recover --keyring-backend test
```
You now are running a validator with a majority of the voting power with the same mainnet state as when you took the snapshot.
You now are running a validator with a majority of the voting power with the same state as mainnet state (at the time you took the snapshot)
10. On your host machine, you can now query the state-exported testnet:
10. On your host machine, you can now query the state export testnet like so:
```sh
osmosisd status
```
11. Here is an example command to ensure complete understanding:
```sh
osmosisd tx bank send wallet osmo1nyphwl8p5yx6fxzevjwqunsfqpcxukmtk8t60m 10000000uosmo --chain-id testing1 --keyring-backend test
```
12. To stop the container and remove its data:
```sh
make localnet-remove-state-export
```
Note: At some point, all the validators (except yours) will get jailed at the same block due to them being offline. When this happens, it make take a little bit of time to process. Once all validators are jailed, you will continue to hit blocks as you did before. If you are only running the validator for a short period of time (< 24 hours) you will not experience this.
Note: At some point, all the validators (except yours) will get jailed at the same block due to them being offline.
When this happens, it may take a little bit of time to process. Once all validators are jailed, you will continue to hit blocks as you did before.
If you are only running the validator for a short period of time (< 24 hours) you will not experience this.
## Accounts
......
tests/localosmosis/mainnet_state/Dockerfile-stateExport deleted 100644 → 0
+ 0
- 62
View file @ 28602791
# syntax=docker/dockerfile:1
# --------------------------------------------------------
# Build
# --------------------------------------------------------
FROM golang:1.18.2-alpine3.15 as build
RUN set -eux; apk add --no-cache ca-certificates build-base;
RUN apk add git
# Needed by github.com/zondax/hid
RUN apk add linux-headers
WORKDIR /osmosis
COPY . /osmosis
# CosmWasm: see https://github.com/CosmWasm/wasmvm/releases
ADD https://github.com/CosmWasm/wasmvm/releases/download/v1.0.0/libwasmvm_muslc.aarch64.a /lib/libwasmvm_muslc.aarch64.a
ADD https://github.com/CosmWasm/wasmvm/releases/download/v1.0.0/libwasmvm_muslc.x86_64.a /lib/libwasmvm_muslc.x86_64.a
RUN sha256sum /lib/libwasmvm_muslc.aarch64.a | grep 7d2239e9f25e96d0d4daba982ce92367aacf0cbd95d2facb8442268f2b1cc1fc
RUN sha256sum /lib/libwasmvm_muslc.x86_64.a | grep f6282df732a13dec836cda1f399dd874b1e3163504dbd9607c6af915b2740479
# CosmWasm: copy the right library according to architecture. The final location will be found by the linker flag `-lwasmvm_muslc`
RUN cp /lib/libwasmvm_muslc.$(uname -m).a /lib/libwasmvm_muslc.a
RUN BUILD_TAGS=muslc LINK_STATICALLY=true make build
# --------------------------------------------------------
# Runner
# --------------------------------------------------------
FROM ubuntu
COPY --from=build /osmosis/build/osmosisd /bin/osmosisd
COPY /tests/localosmosis/mainnet_state/statesync.sh /osmosis/statesync.sh
COPY /tests/localosmosis/mainnet_state/testnetify.py /osmosis/testnetify.py
COPY /tests/localosmosis/testnet_genesis.json /osmosis/testnet_genesis.json
ENV HOME /osmosis
WORKDIR $HOME
# RUN apk update
# RUN apk add jq
# RUN apk add moreutils
# RUN rm -rf /var/cache/apk/*
# RUN apk add --no-cache python3 py3-pip
# RUN apt-get update && apt-get install -y software-properties-common gcc && \
# add-apt-repository -y ppa:deadsnakes/ppa
# RUN apt-get update && apt-get install -y python3.6 python3-distutils python3-pip python3-apt
RUN apt-get update && apt-get install -y python3
RUN chmod +x /osmosis/statesync.sh
RUN /osmosis/statesync.sh
ARG ID=localosmosis
RUN python3 /osmosis/testnetify.py --chain-id=$ID
RUN cp testnet_genesis.json .osmosisd/config/genesis.json
EXPOSE 26656
EXPOSE 26657
EXPOSE 1317
ENTRYPOINT ["osmosisd"]
CMD ["start", "--x-crisis-skip-assert-invariants"]
tests/localosmosis/mainnet_state/docker-compose-state-export.yml deleted 100644 → 0
+ 0
- 15
View file @ 28602791
version: "3"
services:
osmosisd:
image: local:osmosis-se
user: "root:root"
command:
- start
- --x-crisis-skip-assert-invariants
- --rpc.laddr=tcp://0.0.0.0:26657
ports:
- "26657:26657"
- "1317:1317"
- "9090:9090"
- "9091:9091"
tests/localosmosis/mainnet_state/statesync.sh deleted 100755 → 0
+ 0
- 7
View file @ 28602791
#!/bin/bash
# initialize osmosis
osmosisd init --chain-id=localosmosis val
# remove seeds
sed -i.bak -E 's#^(seeds[[:space:]]+=[[:space:]]+).*$#\1""#' ~/.osmosisd/config/config.toml
sed -i.bak -E 's#^(fast_sync[[:space:]]+=[[:space:]]+).*$#\1false#' ~/.osmosisd/config/config.toml
tests/localosmosis/mainnet_state/testnetify.py deleted 100644 → 0
+ 0
- 347
View file @ 28602791
import json
import subprocess
import re, shutil, tempfile
from datetime import datetime
import argparse
import sys
from sys import argv
class CustomHelpFormatter(argparse.HelpFormatter):
def _format_action_invocation(self, action):
if not action.option_strings or action.nargs == 0:
return super()._format_action_invocation(action)
return ', '.join(action.option_strings)
def _split_lines(self, text, width):
if text.startswith('R|'):
return text[2:].splitlines()
# this is the RawTextHelpFormatter._split_lines
return argparse.HelpFormatter._split_lines(self, text, width)
fmt = lambda prog: CustomHelpFormatter(prog,max_help_position=30)
parser = argparse.ArgumentParser(description="Osmosis Installer",formatter_class=fmt)
choice = parser.add_argument_group('Choices')
choice.add_argument(
'-c',
'--chain-id',
type = str,
default="localosmosis",
help='R|Chain ID for newly created testnet \nDefault: localosmosis\n ',
dest="chainID")
if not len(sys.argv) > 1:
parser.set_defaults(chainID="localosmosis")
args = parser.parse_args()
#get values from your priv_validator_key.json to later switch with high power validator
daemon_name = "osmosisd"
#get bas64
result = subprocess.run([daemon_name,"tendermint","show-validator"], stdout=subprocess.PIPE, stderr=subprocess.PIPE, text=True)
base64 = result.stdout.strip()
##base64 = '{"@type":"/cosmos.crypto.ed25519.PubKey","key":"3QVAkiUIkKR3B6kkbd+QqzWDdcExoggbZV5fwH4jKDs="}'
#get validator cons pubkey
val_pubkey = base64[base64.find('key":') +6 :-2]
##val_pubkey = "3QVAkiUIkKR3B6kkbd+QqzWDdcExoggbZV5fwH4jKDs="
#osmosisd debug pubkey {base64} to get address
debug_pubkey = subprocess.run([daemon_name,"debug", "pubkey", base64], stdout=subprocess.PIPE, stderr=subprocess.PIPE, text=True)
#address
address = debug_pubkey.stderr[9: debug_pubkey.stderr.find("\n")]
##based on show-valdiator
##address = "214D831D6F49A75F9104BDC3F2E12A6CC1FC5669"
#feed address into osmosisd debug addr {address} to get bech32 validator address (osmovaloper)
bech32 = subprocess.run([daemon_name,"debug", "addr", address], stdout=subprocess.PIPE, stderr=subprocess.PIPE, text=True)
#osmovaloper
bech32_val = bech32.stderr[bech32.stderr.find("Val: ") + 5: -1]
##operator address
##bech32_val = "osmovaloper1y9xcx8t0fxn4lygyhhpl9cf2dnqlc4nf4pymm4"
#pass osmovaloper address into osmosisd debug bech32-convert -p osmovalcons
bech32_convert = subprocess.run([daemon_name,"debug", "bech32-convert", bech32_val, "-p", "osmovalcons"], stdout=subprocess.PIPE, stderr=subprocess.PIPE, text=True)
#osmovalcons
final_address = bech32_convert.stderr[:bech32_convert.stderr.find("\n")]
##osmovalcons is taken from show-validator
##final_address = "osmovalcons1y9xcx8t0fxn4lygyhhpl9cf2dnqlc4nfpjh8h5"
#own opp address
#exchange the op_address and op_pubkey with own address and pubkey or use above mnemonic for following address
#bottom loan skill merry east cradle onion journey palm apology verb edit desert impose absurd oil bubble sweet glove shallow size build burst effort
#CAN MODIFY
op_address = "osmo12smx2wdlyttvyzvzg54y2vnqwq2qjateuf7thj"
#own pub key
#op_base64_pre = subprocess.run(["osmosisd","query", "auth", "account", op_address], stdout=subprocess.PIPE, stderr=subprocess.PIPE, text=True)
#op_pubkey = op_base64_pre.stdout[op_base64_pre.stdout.find("key: ")+5:op_base64_pre.stdout.find("sequence")-1]
#CAN MODIFY
op_pubkey = "A2MR6q+pOpLtdxh0tHHe2JrEY2KOcvRogtLxHDHzJvOh"
#feed address into osmosisd debug addr {address} to get bech32 validator op address (osmovaloper)
# bech32_op = subprocess.run([daemon_name,"debug", "addr", op_address], stdout=subprocess.PIPE, stderr=subprocess.PIPE, text=True)
#osmovaloper
# bech32_valoper = bech32_op.stderr[bech32_op.stderr.find("Val: ") + 5: -1]
# osmovaloper12smx2wdlyttvyzvzg54y2vnqwq2qjatex7kgq4
def sed_inplace(filename, pattern, repl):
'''
Perform the pure-Python equivalent of in-place `sed` substitution: e.g.,
`sed -i -e 's/'${pattern}'/'${repl}' "${filename}"`.
'''
# For efficiency, precompile the passed regular expression.
pattern_compiled = re.compile(pattern)
# For portability, NamedTemporaryFile() defaults to mode "w+b" (i.e., binary
# writing with updating)
with tempfile.NamedTemporaryFile(mode='w', delete=False) as tmp_file:
with open(filename) as src_file:
for line in src_file:
tmp_file.write(pattern_compiled.sub(repl, line))
# Overwrite the original file with the munged temporary file in a
# manner preserving file attributes (e.g., permissions).
shutil.copystat(filename, tmp_file.name)
shutil.move(tmp_file.name, filename)
#validator cons pubkey: val_pubkey
#osmovalcons: final_address
#validator hex address: address
#osmovaloper: bech32_valoper
#actual account: op_address
#accounts pubkey: op_pubkey
#replace validator cons pubkey (this did not work well due to random forward slashes, did other way later)
#sed_inplace("testnet_genesis.json", "b77zCh/VsRgVvfGXuW4dB+Dhg4PrMWWBC5G2K/qFgiU=", val_pubkey)
#replace validator osmovalcons
print("Replacing osmovalcons1z6skn9g6s7py0klztr7acutr3anqd52k9x5p70 with " + final_address)
sed_inplace("testnet_genesis.json", "osmovalcons1z6skn9g6s7py0klztr7acutr3anqd52k9x5p70", final_address)
#replace validator hex address
print("Replacing 16A169951A878247DBE258FDDC71638F6606D156 with " + address)
sed_inplace("testnet_genesis.json", "16A169951A878247DBE258FDDC71638F6606D156", address)
#replace validator osmovaloper
#print("Replacing osmovaloper1cyw4vw20el8e7ez8080md0r8psg25n0cq98a9n with " + bech32_valoper)
#sed_inplace("testnet_genesis.json", "osmovaloper1cyw4vw20el8e7ez8080md0r8psg25n0cq98a9n", bech32_valoper)
#replace actual account
print("Replacing osmo1cyw4vw20el8e7ez8080md0r8psg25n0c6j07j5 with " + op_address)
sed_inplace("testnet_genesis.json", "osmo1cyw4vw20el8e7ez8080md0r8psg25n0c6j07j5", op_address)
#replace actual account pubkey
print("Replacing AqlNb1FM8veQrT4/apR5B3hww8VApc0LTtZnXhq7FqG0 with " + op_pubkey)
sed_inplace("testnet_genesis.json", "AqlNb1FM8veQrT4/apR5B3hww8VApc0LTtZnXhq7FqG0", op_pubkey)
#open genesis json file with read write priv, load json
test_gen = open("testnet_genesis.json", "r+")
read_test_gen = json.loads(test_gen.read())
#print(read_test_gen.keys())
#change chain-id
print("Current chain-id is " + read_test_gen['chain_id'])
#CAN MODIFY
if args.chainID:
new_chain_id = args.chainID
else:
new_chain_id = "localosmosis"
read_test_gen['chain_id'] = new_chain_id
print("New chain-id is " + read_test_gen['chain_id'])
#validator pubkey must be replaced from b77zCh/VsRgVvfGXuW4dB+Dhg4PrMWWBC5G2K/qFgiU= in two locations
#first under read_test_gen['app_state']['staking']['validators']
#second under read_test_gen['validators']
#i tried to do this using sed_inplace above but the multiple slashes broke it so did this instead
#first val list index
app_state_val_list = read_test_gen['app_state']['staking']['validators']
val_index = [i for i, elem in enumerate(app_state_val_list) if 'Sentinel' in elem['description']['moniker']][0]
#first val list update key
#based on val
app_state_val_list[val_index]['consensus_pubkey']['key'] = val_pubkey
#also update delegator shares and tokens
current_del_share = str(app_state_val_list[val_index]['delegator_shares'])
print("Current delegator shares is " + current_del_share)
app_state_val_list[val_index]['delegator_shares'] = str(int(float(app_state_val_list[val_index]['delegator_shares']) + 1000000000000000)) + ".000000000000000000"
print("New delegator shares is " + app_state_val_list[val_index]['delegator_shares'])
print("Current delegator tokens is " + app_state_val_list[val_index]['tokens'])
app_state_val_list[val_index]['tokens'] = str(int(app_state_val_list[val_index]['tokens']) + 1000000000000000)
print("New delegator tokens is " + app_state_val_list[val_index]['tokens'])
#second val list index
val_list_2 = read_test_gen['validators']
val_list_2_index = [i for i, elem in enumerate(val_list_2) if 'Sentinel' in elem['name']][0]
#second val list update key
#based on val
val_list_2[val_list_2_index]['pub_key']['value'] = val_pubkey
#distribution module fix
dist_address = "osmo1jv65s3grqf6v6jl3dp4t6c9t9rk99cd80yhvld"
app_state_balances_list = read_test_gen['app_state']['bank']['balances']
dist_index = [i for i, elem in enumerate(app_state_balances_list) if dist_address in elem['address']][0]
dist_all = app_state_balances_list[dist_index]['coins']
osmo_index = [i for i, elem in enumerate(dist_all) if 'uosmo' in elem['denom']][0]
current_dist_osmo_bal = dist_all[osmo_index]['amount']
dist_offset_amt = 2
print("Current distribution account uosmo balance is " + current_dist_osmo_bal)
new_dist_osmo_bal = str(int(current_dist_osmo_bal) - dist_offset_amt)
print("New distribution account uosmo balance is " + new_dist_osmo_bal)
dist_all[osmo_index]['amount'] = new_dist_osmo_bal
#change self delegation amount on operator address
#first location
app_state_del_list = read_test_gen['app_state']['staking']['delegations']
del_index = [i for i, elem in enumerate(app_state_del_list) if op_address in elem['delegator_address']][0]
#first val list update share (add 1 BN)
current_share = app_state_del_list[del_index]['shares']
print("Current self delegation is " + str(current_share))
new_share = str(int(float(current_share)) + 1000000000000000)+".000000000000000000"
print("New self delegation is " + new_share)
app_state_del_list[del_index]['shares'] = new_share
#second location
app_state_dist_list = read_test_gen['app_state']['distribution']['delegator_starting_infos']
dist_index = [i for i, elem in enumerate(app_state_dist_list) if op_address in elem['delegator_address']][0]
#second val list update stake (add 1 BN)
current_stake = app_state_dist_list[dist_index]['starting_info']['stake']
print("Current stake is " + str(current_stake))
new_stake = str(int(float(current_stake)) + 1000000000000000)+".000000000000000000"
print("New stake is " + new_stake)
app_state_dist_list[dist_index]['starting_info']['stake'] = new_stake
#get index of val power
val_power_list = read_test_gen['validators']
val_power_index = [i for i, elem in enumerate(val_power_list) if 'Sentinel' in elem['name']][0]
#change val power (add 1 BN)
current_power = int(val_power_list[val_power_index]['power'])
print("Current validator power is " + str(current_power))
new_power = str(current_power + 1000000000)
print("New validator power is " + new_power)
val_power_list[val_power_index]['power'] = new_power
#get index of val power in app state (osmovaloper) (bech32_valoper)
last_val_power_list = read_test_gen['app_state']['staking']['last_validator_powers']
#last_val_power_index = [i for i, elem in enumerate(last_val_power_list) if bech32_valoper in elem['address']][0]
last_val_power_index = [i for i, elem in enumerate(last_val_power_list) if 'osmovaloper1cyw4vw20el8e7ez8080md0r8psg25n0cq98a9n' in elem['address']][0]
val_power = int(read_test_gen['app_state']['staking']['last_validator_powers'][last_val_power_index]['power'])
print("Current validator power in second location is " + str(val_power))
new_val_power = str(val_power + 1000000000)
print("New validator power in second location is " + new_val_power)
read_test_gen['app_state']['staking']['last_validator_powers'][last_val_power_index]['power'] = new_val_power
#update last_total_power (last total bonded across all validators, add 1BN)
last_total_power = int(read_test_gen['app_state']['staking']['last_total_power'])
print("Current last total power is " + str(last_total_power))
new_last_total_power = str(last_total_power + 1000000000)
print("New last total power is " + new_last_total_power)
read_test_gen['app_state']['staking']['last_total_power'] = new_last_total_power
#update operator address amount (add 1 BN)
#find wallet index in bank balance
bank_balance_list = read_test_gen['app_state']['bank']['balances']
op_amount_index = [i for i, elem in enumerate(bank_balance_list) if op_address in elem['address']][0]
#get uosmo index from wallet
op_wallet = read_test_gen['app_state']['bank']['balances'][op_amount_index]['coins']
op_uosmo_index = [i for i, elem in enumerate(op_wallet) if 'uosmo' in elem['denom']][0]
#update uosmo amount
op_uosmo = int(op_wallet[op_uosmo_index]['amount'])
print("Current operator address uosmo balance is " + str(op_uosmo))
new_op_uosmo = str(op_uosmo + 1000000000000000)
print("New operator address uosmo balance is " + new_op_uosmo)
op_wallet[op_uosmo_index]['amount'] = new_op_uosmo
#update total OSMO supply (add 2 BN)
#supply list (ibc, ion, osmo)
supply = read_test_gen['app_state']['bank']['supply']
#get index of osmo supply
osmo_index = [i for i, elem in enumerate(supply) if 'uosmo' in elem['denom']][0]
#get osmo supply value
osmo_supply = supply[osmo_index]['amount']
print("Current OSMO supply is " + osmo_supply)
#update osmo supply to new total osmo value (add 2 Billion OSMO)
#subtract by however much module account is subtracted by
osmo_supply_new = int(osmo_supply) + 2000000000000000 - dist_offset_amt
print("New OSMO supply is " + str(osmo_supply_new))
supply[osmo_index]['amount'] = str(osmo_supply_new)
#update bonded_tokens_pool module balance osmo1fl48vsnmsdzcv85q5d2q4z5ajdha8yu3aq6l09 (add 1BN)
#get list of bank balances
bank_bal_list = read_test_gen['app_state']['bank']['balances']
#get index of module account
module_acct_index = [i for i, elem in enumerate(bank_bal_list) if 'osmo1fl48vsnmsdzcv85q5d2q4z5ajdha8yu3aq6l09' in elem['address']][0]
#get current value
module_denom_list = bank_bal_list[module_acct_index]['coins']
osmo_bal_index = [i for i, elem in enumerate(module_denom_list) if 'uosmo' in elem['denom']][0]
osmo_bal = bank_bal_list[module_acct_index]['coins'][osmo_bal_index]['amount']
print("Current bonded tokens pool module account balance is " + osmo_bal)
#increase by 1BN
new_osmo_bal = int(osmo_bal) + 1000000000000000
print("New bonded tokens pool module account balance is " + str(new_osmo_bal))
bank_bal_list[module_acct_index]['coins'][osmo_bal_index]['amount'] = str(new_osmo_bal)
#edit epoch params
#change epoch duration to 21600s
epochs_list = read_test_gen['app_state']['epochs']['epochs'][0]
duration_current = epochs_list['duration']
print("Current epoch duration is " + duration_current)
#21600s for 6 hour epoch
#CAN MODIFY
new_duration = '21600s'
print("New epoch duration is " + new_duration)
epochs_list['duration'] = new_duration
#change current_epoch_start_time
start_time_current = epochs_list['current_epoch_start_time']
print("Current epoch start time is " + start_time_current)
#today = date.today()
now = datetime.now()
#date_format = now.strftime("%Y-%m-%d")
date_format = now.strftime("%Y-%m-%d"+"T"+"%H:%M:")
start_time_current_list = list(start_time_current)
start_time_current_list[:17] = date_format
start_time_new = ''.join(start_time_current_list)
epochs_list['current_epoch_start_time'] = start_time_new
print("New epoch start time is " + start_time_new)
#edit gov params
#change VotingPeriod
current_voting_period = read_test_gen['app_state']['gov']['voting_params']['voting_period']
print("Current voting period is " + current_voting_period)
#180s for 3 minute voting period
#CAN MODIFY
new_voting_period = "180s"
print("New voting period is " + new_voting_period)
read_test_gen['app_state']['gov']['voting_params']['voting_period'] = new_voting_period
print("Please wait while file writes over itself, this may take 60 seconds or more")
#go back to begining of file, write over with new values
test_gen.seek(0)
json.dump(read_test_gen, test_gen)
#delete remainder in case new data is shorter than old
test_gen.truncate()
tests/localosmosis/state_export/docker-compose.yml 0 → 100644
+ 27
- 0
View file @ bf9de2d9
version: "3"
services:
osmosisd:
image: local:osmosis
build:
context: ../../../
dockerfile: Dockerfile
args:
RUNNER_IMAGE: alpine:3.16
GO_VERSION: 1.18
volumes:
- ./scripts/start.sh:/osmosis/start.sh
- ./scripts/testnetify.py:/osmosis/testnetify.py
- ./state_export.json:/osmosis/state_export.json
- ./.osmosisd/:/osmosis/.osmosisd/
entrypoint:
- /osmosis/start.sh
environment:
- MONIKER=val
- CHAIN_ID=localosmosis
ports:
- 26657:26657
- 1317:1317
- 9090:9090
- 9091:9091
tests/localosmosis/state_export/scripts/start.sh 0 → 100755
+ 72
- 0
View file @ bf9de2d9
#!/bin/sh
set -e
set -o pipefail
OSMOSIS_HOME=$HOME/.osmosisd
CONFIG_FOLDER=$OSMOSIS_HOME/config
DEFAULT_MNEMONIC="bottom loan skill merry east cradle onion journey palm apology verb edit desert impose absurd oil bubble sweet glove shallow size build burst effort"
DEFAULT_CHAIN_ID="localosmosis"
DEFAULT_MONIKER="val"
# Override default values with environment variables
MNEMONIC=${MNEMONIC:-$DEFAULT_MNEMONIC}
CHAIN_ID=${CHAIN_ID:-$DEFAULT_CHAIN_ID}
MONIKER=${MONIKER:-$DEFAULT_MONIKER}
install_prerequisites () {
apk add -q --no-cache \
dasel \
python3 \
py3-pip
}
edit_config () {
# Remove seeds
dasel put string -f $CONFIG_FOLDER/config.toml '.p2p.seeds' ''
# Disable fast_sync
dasel put bool -f $CONFIG_FOLDER/config.toml '.fast_sync' 'false'
# Expose the rpc
dasel put string -f $CONFIG_FOLDER/config.toml '.rpc.laddr' "tcp://0.0.0.0:26657"
}
if [[ ! -d $CONFIG_FOLDER ]]
then
install_prerequisites
echo "Chain ID: $CHAIN_ID"
echo "Moniker: $MONIKER"
echo $MNEMONIC | osmosisd init -o --chain-id=$CHAIN_ID --home $OSMOSIS_HOME --recover $MONIKER 2> /dev/null
echo $MNEMONIC | osmosisd keys add my-key --recover --keyring-backend test > /dev/null 2>&1
ACCOUNT_PUBKEY=$(osmosisd keys show --keyring-backend test my-key --pubkey | dasel -r json '.key' --plain)
ACCOUNT_ADDRESS=$(osmosisd keys show -a --keyring-backend test my-key --bech acc)
VALIDATOR_PUBKEY_JSON=$(osmosisd tendermint show-validator --home $OSMOSIS_HOME)
VALIDATOR_PUBKEY=$(echo $VALIDATOR_PUBKEY_JSON | dasel -r json '.key' --plain)
VALIDATOR_HEX_ADDRESS=$(osmosisd debug pubkey $VALIDATOR_PUBKEY_JSON 2>&1 --home $OSMOSIS_HOME | grep Address | cut -d " " -f 2)
VALIDATOR_ACCOUNT_ADDRESS=$(osmosisd debug addr $VALIDATOR_HEX_ADDRESS 2>&1 --home $OSMOSIS_HOME | grep Acc | cut -d " " -f 3)
VALIDATOR_OPERATOR_ADDRESS=$(osmosisd debug addr $VALIDATOR_HEX_ADDRESS 2>&1 --home $OSMOSIS_HOME | grep Val | cut -d " " -f 3)
VALIDATOR_CONSENSUS_ADDRESS=$(osmosisd debug bech32-convert $VALIDATOR_OPERATOR_ADDRESS -p osmovalcons --home $OSMOSIS_HOME 2>&1)
python3 -u testnetify.py \
-i /osmosis/state_export.json \
-o $CONFIG_FOLDER/genesis.json \
-c $CHAIN_ID \
--validator-hex-address $VALIDATOR_HEX_ADDRESS \
--validator-operator-address $VALIDATOR_OPERATOR_ADDRESS \
--validator-consensus-address $VALIDATOR_CONSENSUS_ADDRESS \
--validator-pubkey $VALIDATOR_PUBKEY \
--account-pubkey $ACCOUNT_PUBKEY \
--account-address $ACCOUNT_ADDRESS \
--prune-ibc
edit_config
fi
osmosisd start --home $OSMOSIS_HOME --x-crisis-skip-assert-invariants
tests/localosmosis/state_export/scripts/testnetify.py 0 → 100755
+ 372
- 0
View file @ bf9de2d9
import json
import argparse
from datetime import datetime
from dataclasses import dataclass
# Classes
@dataclass
class Validator:
moniker: str
pubkey: str
hex_address: str
operator_address: str
consensus_address: str
@dataclass
class Account:
pubkey: str
address: str
# Contants
BONDED_TOKENS_POOL_MODULE_ADDRESS = "osmo1fl48vsnmsdzcv85q5d2q4z5ajdha8yu3aq6l09"
DISTRIBUTION_MODULE_ADDRESS = "osmo1jv65s3grqf6v6jl3dp4t6c9t9rk99cd80yhvld"
DISTRIBUTION_MODULE_OFFSET = 2
config = {
"governance_voting_period": "180s",
"epoch_duration": '21600s',
}
def replace(d, old_value, new_value):
"""
Replace all the occurences of `old_value` with `new_value`
in `d` dictionary
"""
for k in d.keys():
if isinstance(d[k], dict):
replace(d[k], old_value, new_value)
elif isinstance(d[k], list):
for i in range(len(d[k])):
if isinstance(d[k][i], dict) or isinstance(d[k][i], list):
replace(d[k][i], old_value, new_value)
else:
if d[k][i] == old_value:
d[k][i] = new_value
else:
if d[k] == old_value:
d[k] = new_value
def replace_validator(genesis, old_validator, new_validator):
replace(genesis, old_validator.hex_address, new_validator.hex_address)
replace(genesis, old_validator.consensus_address, new_validator.consensus_address)
# replace(genesis, old_validator.pubkey, new_validator.pubkey)
for validator in genesis["validators"]:
if validator['name'] == old_validator.moniker:
validator['pub_key']['value'] = new_validator.pubkey
for validator in genesis['app_state']['staking']['validators']:
if validator['description']['moniker'] == old_validator.moniker:
validator['consensus_pubkey']['key'] = new_validator.pubkey
# This creates problems
# replace(genesis, old_validator.operator_address, new_validator.operator_address)
# replacing operator_address in lockup > synthetic_locks
# for synthetic_lock in genesis['app_state']['lockup']['synthetic_locks']:ß
# if synthetic_lock['synth_denom'].endswith(old_validator.operator_address):
# synthetic_lock['synth_denom'] = synthetic_lock['synth_denom'].replace(
# old_validator.operator_address, new_validator.operator_address)
# Replacing operator_address in incentives > gauges
# for gauge in genesis['app_state']['incentives']['gauges']:
# if gauge['distribute_to']['denom'].endswith(old_validator.operator_address):
# gauge['distribute_to']['denom'] = gauge['distribute_to']['denom'].replace(
# old_validator.operator_address, new_validator.operator_address)
def replace_account(genesis, old_account, new_account):
replace(genesis, old_account.address, new_account.address)
replace(genesis, old_account.pubkey, new_account.pubkey)
def create_parser():
parser = argparse.ArgumentParser(
formatter_class=argparse.RawDescriptionHelpFormatter,
description='Create a testnet from a state export')
parser.add_argument(
'-c',
'--chain-id',
type = str,
default="localosmosis",
help='Chain ID for the testnet \nDefault: localosmosis\n'
)
parser.add_argument(
'-i',
'--input',
type = str,
default="state_export.json",
dest='input_genesis',
help='Path to input genesis'
)
parser.add_argument(
'-o',
'--output',
type = str,
default="testnet_genesis.json",
dest='output_genesis',
help='Path to output genesis'
)
parser.add_argument(
'--validator-hex-address',
type = str,
help='Validator hex address to replace'
)
parser.add_argument(
'--validator-operator-address',
type = str,
help='Validator operator address to replace'
)
parser.add_argument(
'--validator-consensus-address',
type = str,
help='Validator consensus address to replace'
)
parser.add_argument(
'--validator-pubkey',
type = str,
help='Validator pubkey to replace'
)
parser.add_argument(
'--account-pubkey',
type = str,
help='Account pubkey to replace'
)
parser.add_argument(
'--account-address',
type = str,
help='Account address to replace'
)
parser.add_argument(
'-q',
'--quiet',
action='store_false',
help='Less verbose output'
)
parser.add_argument(
'--prune-ibc',
action='store_true',
help='Prune the IBC module'
)
parser.add_argument(
'--pretty-output',
action='store_true',
help='Properly indent output genesis (increases time and file size)'
)
return parser
def main():
parser = create_parser()
args = parser.parse_args()
new_validator = Validator(
moniker = "val",
pubkey = args.validator_pubkey,
hex_address = args.validator_hex_address,
operator_address = args.validator_operator_address,
consensus_address = args.validator_consensus_address
)
old_validator = Validator(
moniker = "Sentinel dVPN",
pubkey = "b77zCh/VsRgVvfGXuW4dB+Dhg4PrMWWBC5G2K/qFgiU=",
hex_address = "16A169951A878247DBE258FDDC71638F6606D156",
operator_address = "osmovaloper1cyw4vw20el8e7ez8080md0r8psg25n0cq98a9n",
consensus_address = "osmovalcons1z6skn9g6s7py0klztr7acutr3anqd52k9x5p70"
)
new_account = Account(
pubkey = args.account_pubkey,
address = args.account_address
)
old_account = Account(
pubkey = "AqlNb1FM8veQrT4/apR5B3hww8VApc0LTtZnXhq7FqG0",
address = "osmo1cyw4vw20el8e7ez8080md0r8psg25n0c6j07j5"
)
print("📝 Opening {}... (it may take a while)".format(args.input_genesis))
with open(args.input_genesis, 'r') as f:
genesis = json.load(f)
# Replace chain-id
if args.verbose:
print("🔗 Replace chain-id {} with {}".format(genesis['chain_id'], args.chain_id))
genesis['chain_id'] = args.chain_id
# Update gov module
if args.verbose:
print("🗳️ Update gov module")
print("\tModify governance_voting_period from {} to {}".format(
genesis['app_state']['gov']['voting_params']['voting_period'],
config["governance_voting_period"]))
genesis['app_state']['gov']['voting_params']['voting_period'] = config["governance_voting_period"]
# Update epochs module
if args.verbose:
print("⌛ Update epochs module")
print("\tModify epoch_duration from {} to {}".format(
genesis['app_state']['epochs']['epochs'][0]['duration'],
config["epoch_duration"]))
print("\tReset current_epoch_start_time")
genesis['app_state']['epochs']['epochs'][0]['duration'] = config["epoch_duration"]
genesis['app_state']['epochs']['epochs'][0]['current_epoch_start_time'] = datetime.now().isoformat() + 'Z'
# Prune IBC
if args.prune_ibc:
if args.verbose:
print("🕸 Pruning IBC module")
genesis['app_state']["ibc"]["channel_genesis"]["ack_sequences"] = []
genesis['app_state']["ibc"]["channel_genesis"]["acknowledgements"] = []
genesis['app_state']["ibc"]["channel_genesis"]["channels"] = []
genesis['app_state']["ibc"]["channel_genesis"]["commitments"] = []
genesis['app_state']["ibc"]["channel_genesis"]["receipts"] = []
genesis['app_state']["ibc"]["channel_genesis"]["recv_sequences"] = []
genesis['app_state']["ibc"]["channel_genesis"]["send_sequences"] = []
genesis['app_state']["ibc"]["client_genesis"]["clients"] = []
genesis['app_state']["ibc"]["client_genesis"]["clients_consensus"] = []
genesis['app_state']["ibc"]["client_genesis"]["clients_metadata"] = []
# Impersonate validator
if args.verbose:
print("🚀 Replace validator")
# print("\t{:50} -> {}".format(old_validator.moniker, new_validator.moniker))
print("\t{:20} {}".format("Pubkey", new_validator.pubkey))
print("\t{:20} {}".format("Consensus address", new_validator.consensus_address))
print("\t{:20} {}".format("Operator address", new_validator.operator_address))
print("\t{:20} {}".format("Hex address", new_validator.hex_address))
replace_validator(genesis, old_validator, new_validator)
# Impersonate account
if args.verbose:
print("🧪 Replace account")
print("\t{:20} {}".format("Pubkey", new_account.pubkey))
print("\t{:20} {}".format("Address", new_account.address))
replace_account(genesis, old_account, new_account)
# Update staking module
if args.verbose:
print("🥩 Update staking module")
# Replace validator pub key in genesis['app_state']['staking']['validators']
for validator in genesis['app_state']['staking']['validators']:
if validator['description']['moniker'] == old_validator.moniker:
# Update delegator shares
validator['delegator_shares'] = str(int(float(validator['delegator_shares']) + 1000000000000000)) + ".000000000000000000"
print("\tUpdate delegator shares to {}".format(validator['delegator_shares']))
# Update tokens
validator['tokens'] = str(int(validator['tokens']) + 1000000000000000)
print("\tUpdate tokens to {}".format(validator['tokens']))
break
# Update self delegation on operator address
for delegation in genesis['app_state']['staking']['delegations']:
if delegation['delegator_address'] == new_account.address:
# delegation['validator_address'] = new_validator.operator_address
delegation['shares'] = str(int(float(delegation['shares'])) + 1000000000000000) + ".000000000000000000"
print("\tUpdate {} delegation shares to {} to {}".format(new_account.address, delegation['validator_address'], delegation['shares']))
break
# Update genesis['app_state']['distribution']['delegator_starting_infos'] on operator address
for delegator_starting_info in genesis['app_state']['distribution']['delegator_starting_infos']:
if delegator_starting_info['delegator_address'] == new_account.address:
delegator_starting_info['starting_info']['stake'] = str(int(float(delegator_starting_info['starting_info']['stake']) + 1000000000000000))+".000000000000000000"
print("\tUpdate {} stake to {}".format(delegator_starting_info['delegator_address'], delegator_starting_info['starting_info']['stake']))
break
if args.verbose:
print("🔋 Update validator power")
# Update power in genesis["validators"]
for validator in genesis["validators"]:
if validator['name'] == old_validator.moniker:
validator['power'] = str(int(validator['power']) + 1000000000)
print("\tUpdate {} validator power to {}".format(validator['address'], validator['power']))
break
for validator_power in genesis['app_state']['staking']['last_validator_powers']:
if validator_power['address'] == old_validator.operator_address:
validator_power['power'] = str(int(validator_power['power']) + 1000000000)
if args.verbose:
print("\tUpdate {} last_validator_power to {}".format(old_validator.operator_address, validator_power['power']))
break
# Update total power
genesis['app_state']['staking']['last_total_power'] = str(int(genesis['app_state']['staking']['last_total_power']) + 1000000000)
if args.verbose:
print("\tUpdate last_total_power to {}".format(genesis['app_state']['staking']['last_total_power']))
# Update bank module
if args.verbose:
print("💵 Update bank module")
for balance in genesis['app_state']['bank']['balances']:
if balance['address'] == new_account.address:
for coin in balance['coins']:
if coin['denom'] == "uosmo":
coin["amount"] = str(int(coin["amount"]) + 1000000000000000)
print("\tUpdate {} uosmo balance to {}".format(new_account.address, coin["amount"]))
break
break
# Add 1 BN uosmo to bonded_tokens_pool module address
for balance in genesis['app_state']['bank']['balances']:
if balance['address'] == BONDED_TOKENS_POOL_MODULE_ADDRESS:
# Find uosmo
for coin in balance['coins']:
if coin['denom'] == "uosmo":
coin["amount"] = str(int(coin["amount"]) + 1000000000000000)
print("\tUpdate {} (bonded_tokens_pool_module) uosmo balance to {}".format(BONDED_TOKENS_POOL_MODULE_ADDRESS, coin["amount"]))
break
break
# Distribution module fix
for balance in genesis['app_state']['bank']['balances']:
if balance['address'] == DISTRIBUTION_MODULE_ADDRESS:
# Find uosmo
for coin in balance['coins']:
if coin['denom'] == "uosmo":
coin["amount"] = str(int(coin["amount"]) - DISTRIBUTION_MODULE_OFFSET)
print("\tUpdate {} (distribution_module) uosmo balance to {}".format(DISTRIBUTION_MODULE_ADDRESS, coin["amount"]))
break
break
# Update bank balance
for supply in genesis['app_state']['bank']['supply']:
if supply["denom"] == "uosmo":
print("\tUpdate total uosmo supply from {} to {}".format(supply["amount"], str(int(supply["amount"]) + 2000000000000000 - DISTRIBUTION_MODULE_OFFSET)))
supply["amount"] = str(int(supply["amount"]) + 2000000000000000 - DISTRIBUTION_MODULE_OFFSET)
break
print("📝 Writing {}... (it may take a while)".format(args.output_genesis))
with open(args.output_genesis, 'w') as f:
if args.pretty_output:
f.write(json.dumps(genesis, indent=2))
else:
f.write(json.dumps(genesis))
if __name__ == '__main__':
main()
\ No newline at end of file
Supports Markdown
0% or .
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment