How to build portainer in production

GigaDB Portainer service What is portainer? Portainer is an open-source management UI for Docker, including Docker Swarm environment. Portainer makes it easier for you to manage your Docker containers, it allows you to manage containers, images, networks, and volumes from the web-based Portainer dashboard. Pre-requisites The staging server and live server are up and running. Here and here are the details of how to provision and configure an EC2 server using Ansible and Terraform. The Let’s Encrypt certificate fallout has been fixed by getting the latest code from this PR #198. Have Docker Hub account, and store DOCKER_HUB_USERNAME and DOCKER_HUB_PASSWORD which is the access token in gitlab CI/CD variables. Steps to configure the portainer, details can be found at this PR #201 Changes to the DNS record Create an A record for access portainer on staging and on live as following: Record name Type IP portainer.$staging_url A staging server IP portainer.$live_url A live server IP Changes to the gitlab variables Create a new variable as following: Key Value Environment PORTAINER_PASSWORD “Password” All(default) DOCKER_HUB_USERNAME “User name” All(default) DOCKER_HUB_PASSWORD “Access token” All(default) Variable PORTAINER_BCRYPT (if any) could be deleted. ...

November 15, 2021 · 2 min · 327 words · Ken Cho

How to add portainer for monitoring dockers

Task Make portainer on staging and live deployment accessible with HTTPS on default port #790 Add portainer container service on staging and live deployment #791 PR #201 Steps A. Add portainer service available locally Update docker-compose.yml portainer: image: portainer/portainer-ce:latest volumes: - /var/run/docker.sock:/var/run/docker.sock - portainer_data:/data ports: - 9009:9000 - 8008:8000 command: -H unix:///var/run/docker.sock --admin-password $PORTAINER_BCRYPT volumes: le_config: le_webrootpath: portainer_data: Create PORTAINER_PASSWORD in gitlab variables Create start_portainer.sh, if-else block to avoid keep generating PORTAINER_BCRYPT in .env #!/usr/bin/env bash # bail out as soon as there is an error set -eux # Load environment variables source "./.env" source "./.secrets" # docker-compose executable if [[ $GIGADB_ENV != "dev" && $GIGADB_ENV != "CI" ]];then DOCKER_COMPOSE="docker-compose --tlsverify -H=$REMOTE_DOCKER_HOST -f ops/deployment/docker-compose.production-envs.yml" else DOCKER_COMPOSE="docker-compose" fi if ! [ -z "${PORTAINER_BCRYPT+x}" ];then echo "PORTAINER_BCRYPT value has been set in .env already" # start portainer in detached mode and make sure volume are recreated (rather than use potential previous state that my be erroneous) $DOCKER_COMPOSE up --detach --renew-anon-volumes portainer else echo "PORTAINER_BCRYPT value is empty" echo "Generate bcrypt from password" P_BCRYPT=$(docker run --rm httpd:2.4-alpine htpasswd -nbB admin $PORTAINER_PASSWORD | cut -d ":" -f 2 | sed -e 's/\$/\\\$/g') echo "PORTAINER_BCRYPT=$P_BCRYPT" >> .env # start portainer in detached mode and make sure volume are recreated (rather than use potential previous state that my be erroneous) $DOCKER_COMPOSE up --detach --renew-anon-volumes portainer fi Update up.sh to only start portainer in MacOS # start the container admin UI (not in CI) if [ "$(uname)" == "Darwin" ];then ./ops/scripts/start_portainer.sh fi; Spin up all containers kencho/gigadb-website % ./up.sh Test the http response using curl % curl -I localhost:9009 HTTP/1.1 200 OK Accept-Ranges: bytes Cache-Control: max-age=31536000 Content-Length: 6176 Content-Type: text/html; charset=utf-8 Last-Modified: Sun, 10 Oct 2021 23:45:45 GMT X-Content-Type-Options: nosniff X-Xss-Protection: 1; mode=block Date: Thu, 21 Oct 2021 08:14:01 GMT Go to http://localhost:9009/ and have fun! Reference Heavily adapted form here ...

October 21, 2021 · 2 min · 304 words · Ken Cho

To resolve XDebug dependency problem

Problem ---> Running in 4c7fe5b9cf5e pecl/xdebug requires PHP (version >= 7.2.0, version <= 8.0.99), installed version is 7.1.30 No valid packages found install failed ERROR: Service 'test' failed to build : The command '/bin/sh -c if [ ${INSTALL_XDEBUG} = true ]; then if [ $(php -r "echo PHP_MAJOR_VERSION;") = "5" ]; then pecl install xdebug-2.5.5; else pecl install xdebug; fi && docker-php-ext-enable xdebug ;fi' returned a non-zero code: 1 Effect The docker-compose run --rm test cannot be run. Then the test service cannot be built. ...

January 7, 2021 · 2 min · 216 words · Ken Cho

Commands use to run gigadb

To start a local gigaDB server docker-compose run --rm webapp Link to test if the web page is up If there are some build errors Rebuild and up the webapp containers again. docker-compose run --rm webapp If not work, restart the docker container app and rebuild the containers. If not work, check if psql, gigadb, database is presented in ~/.containers/-data and remove it. rm -rf ~/.containers-data/gigadb Run docker-compose run --rm webapp again to rebuild the containers. How to remove images List all images docker images -a Remove all images docker rmi $(docker images -a -q) How to remove container List all containers docker ps -a Stop the containers docker stop $(docker ps -a -q) Remove stopped containers docker rm $(docker ps -a -q) source digital ocean How to run a functional test in a docker environment for testing? Write unit test script and place in protected/tests/functional-test folder. Change the unit_functional script ./bin/phpunit protected/tests/functional-test --verbose --configuration protected/tests/phpunit.xml --no-coverage Run specific unit test: docker-compose run --rm test ./tests/unit_functional Output will be like this: PHPUnit 5.7.27 by Sebastian Bergmann and contributors. Runtime: PHP 7.1.30 with Xdebug 2.9.6 Configuration: /var/www/protected/tests/phpunit.xml ... 3 / 3 (100%) Time: 7.15 seconds, Memory: 14.00MB OK (3 tests, 7 assertions) How to run a functional test on a single file in docker environment? docker oneliner docker-compose run --rm test ./bin/phpunit --testsuite functional --bootstrap protected/tests/bootstrap.php --verbose --configuration protected/tests/phpunit.xml --no-coverage protected/tests/functional/AdminSiteAccessTest.php Or go into docker bash and run phpunit docker-compose run --rm test bash root@45a46a8441d3:/var/www# ./bin/phpunit --testsuite functional --bootstrap protected/tests/bootstrap.php --verbose --configuration protected/tests/phpunit.xml --no-coverage protected/tests/functional/AdminSiteAccessTest.php root@45a46a8441d3:/var/www# ./bin/phpunit --testsuite functional --bootstrap protected/tests/bootstrap.php --verbose --configuration protected/tests/phpunit.xml --no-coverage protected/tests/functional/AdminSiteAccessTest.php PHPUnit 5.7.27 by Sebastian Bergmann and contributors. Runtime: PHP 7.1.30 with Xdebug 2.9.6 Configuration: /var/www/protected/tests/phpunit.xml ... 3 / 3 (100%) Time: 4.18 seconds, Memory: 14.00MB OK (3 tests, 7 assertions) How to run a Behat test in a docker environment for testing Update the scenario syntax specific *.feature file, like: @ok @files Scenario: Files - Call to Actions Given I am not logged in to Gigadb web site When I go to "/dataset/101001" And I follow "Files" Then I should see a link "(FTP site)" to "ftp://climb.genomics.cn/pub/10.5524/101001_102000/101001/" with title "FTP site" Then I should see a button "Table Settings" Check if related functions in features/bootstrap/*.php needed to be updated, eg: public function iShouldSeeTabWithTable($arg1, TableNode $table) { if ("Funding" == $arg1) { $this->minkContext->getSession()->getPage()->clickLink($arg1); //| Funding body | Awardee | Award ID | Comments | foreach($table as $row) { PHPUnit_Framework_Assert::assertTrue( $this->minkContext->getSession()->getPage()->hasContent($row['Funding body']) ); PHPUnit_Framework_Assert::assertTrue( $this->minkContext->getSession()->getPage()->hasContent($row['Awardee']) ); PHPUnit_Framework_Assert::assertTrue( $this->minkContext->getSession()->getPage()->hasContent($row['Award ID']) ); PHPUnit_Framework_Assert::assertTrue( $this->minkContext->getSession()->getPage()->hasContent($row['Comments']) ); } } elseif("Files" == $arg1) { //| File name | Sample ID | Data Type | File Format | Size | Release date | link | foreach($table as $row) { $link = $row['link']; PHPUnit_Framework_Assert::assertTrue( $this->minkContext->getSession()->getPage()->hasContent($row['File name']), "File name match" ); PHPUnit_Framework_Assert::assertTrue( $this->minkContext->getSession()->getPage()->hasContent($row['Sample ID']), "Sample ID match" ); PHPUnit_Framework_Assert::assertTrue( $this->minkContext->getSession()->getPage()->hasContent($row['Data Type']), "Data Type match" ); PHPUnit_Framework_Assert::assertTrue( $this->minkContext->getSession()->getPage()->hasContent($row['File Format']), "File Format match" ); PHPUnit_Framework_Assert::assertTrue( $this->minkContext->getSession()->getPage()->hasContent($row['Size']), "Size match" ); PHPUnit_Framework_Assert::assertTrue( $this->minkContext->getSession()->getPage()->hasContent($row['Release date']), "Release date match" ); if ($link) { $this->minkContext->assertSession()->elementExists('css',"a.download-btn[href='$link']"); } } } elseif("Sample" == $arg1) { //| Sample ID | Common Name | Scientific Name | Sample Attributes | Taxonomic ID | Genbank Name | foreach($table as $row) { PHPUnit_Framework_Assert::assertTrue( $this->minkContext->getSession()->getPage()->hasContent($row['Sample ID']), "Sample ID match" ); if($row['Common Name']) { PHPUnit_Framework_Assert::assertTrue( $this->minkContext->getSession()->getPage()->hasContent($row['Common Name']), "Common Name match" ); } PHPUnit_Framework_Assert::assertTrue( $this->minkContext->getSession()->getPage()->hasContent($row['Scientific Name']), "Scientific Name match" ); PHPUnit_Framework_Assert::assertTrue( $this->minkContext->getSession()->getPage()->hasContent($row['Sample Attributes']), "Sample Attributes match" ); PHPUnit_Framework_Assert::assertTrue( $this->minkContext->getSession()->getPage()->hasContent($row['Taxonomic ID']), "Taxonomic ID match" ); PHPUnit_Framework_Assert::assertTrue( $this->minkContext->getSession()->getPage()->hasContent($row['Genbank Name']), "Genbank Name match" ); } } else { PHPUnit_Framework_Assert::fail("Unknown type of tab"); } } Assign @wip tag to the working scenario in .feature file, this can smooth the running of behat test. @wip @files Scenario: Files - Call to Actions Given I am not logged in to Gigadb web site When I go to "/dataset/101001" Run the Behat test command on specific scenario. docker-compose run --rm test bin/behat -vv --stop-on-failure --tags @wip ...

July 24, 2020 · 8 min · 1525 words · Ken Cho

Learn, todo and done 22-26/6/20

Learn Docker TDD Codeception PHPUnit Yii2 Todo Done Issue #291 Review PR#452 Issue #423 Issue #425 Issue #440 Reference

June 22, 2020 · 1 min · 19 words · Ken Cho