Commit 4e24914b authored by Bartłomiej Górnicki's avatar Bartłomiej Górnicki

Merge branch 'autoclave'

parents 7ca4d746 03c25e2f
stages:
- build_staging
- restart_staging
- clean_up
build_staging:
stage: build_staging
script:
- echo "Building current develop branch"
- ./run.sh build stg
environment:
name: staging
only:
- develop
restart_staging:
stage: restart_staging
script:
- echo "Stopping current condenser image"
- ./run.sh stop stg
- echo "Starting latest condenser image"
- ./run.sh start stg
dependencies:
- build_staging
environment:
name: staging
only:
- develop
clean_up:
stage: clean_up
script:
- echo "Cleanup filespace / unused images / containers / networks"
- docker image prune -f
- docker container prune -f
- docker network prune -f
dependencies:
- restart_staging
environment:
name: staging
only:
- develop
......@@ -13,7 +13,7 @@ RUN npm install -g yarn
WORKDIR /var/app
RUN mkdir -p /var/app
ADD package.json yarn.lock /var/app/
RUN yarn install --non-interactive --frozen-lockfile
RUN yarn install --non-interactive --frozen-lockfile --ignore-optional
COPY . /var/app
......
# Condenser
Condenser is the react.js web interface to the world's first and best
blockchain-based social media platform, steemit.com. It uses
[STEEM](https://github.com/steemit/steem), a blockchain powered by DPoS Governance and ChainBase DB to store JSON-based content for a plethora of web
Condenser is the react.js web interface to the
blockchain-based social media platform, Hive.blog. It uses a
[Hive compatible blockchain](https://gitlab.syncad.com/hive/hive), powered by DPoS Governance and ChainBase DB to store JSON-based content for a plethora of web
applications.
## Why would I want to use Condenser (steemit.com front-end)?
## Why would I want to use Condenser?
* Learning how to build blockchain-based web applications using STEEM as a
content storage mechanism in react.js
* Reviewing the inner workings of the steemit.com social media platform
* Assisting with software development for steemit.com
* Learning how to build blockchain-based web applications using Hive compatible blockchain as a content storage mechanism in react.js
* Reviewing the inner workings of the Hive.blog social media platform
* Assisting with software development for Hive.blog and Hive.io
## Installation
#### Docker
We highly recommend using docker to run condenser in production. This is how we run the
live steemit.com site and it is the most supported (and fastest) method of
both building and running condenser. We will always have the latest version
of condenser (master branch) available on Docker Hub. Configuration settings
can be set using environment variables (see configuration section below for
more information). If you need to install docker, you can get it at
https://get.docker.com
To bring up a running container it's as simple as this:
```bash
docker run -it -p 8080:8080 steemit/condenser
```
Environment variables can be added like this:
```bash
docker run -it -p 8080:8080 steemit/condenser
```
If you would like to modify, build, and run condenser using docker, it's as
simple as pulling in the github repo and issuing one command to build it,
like this:
```bash
git clone https://github.com/steemit/condenser
git clone https://gitlab.syncad.com/hive/condenser
cd condenser
docker build -t="myname/condenser:mybranch" .
docker run -it -p 8080:8080 myname/condenser:mybranch
```
#### Docker Compose
If you like to run and build condenser and additionally a reverse-proxy using an Nginx Docker image, with companion Letsencrypt (SSL) support, you can simple launch the Docker-compose files via the included `run.sh`-scripts.
```bash
git clone https://gitlab.syncad.com/hive/condenser
cd condenser
./run.sh start proxy # to start the nginx reverse proxy (with ssl support)
./run.sh start (prod|dev|stg) # to build and start the condensor image
./run.sh logs (prod|dev|stg) # (optionally) to attach to the condensor image and inspect its logs
```
## Building from source without docker (the 'traditional' way):
(better if you're planning to do condenser development)
#### Clone the repository and make a tmp folder
```bash
git clone https://github.com/steemit/condenser
git clone https://gitlab.syncad.com/hive/condenser
cd condenser
mkdir tmp
```
#### Install dependencies
Install at least Node v8.7 if you don't already have it. We recommend using
`nvm` to do this as it's both the simplest way to install and manage
installed version(s) of node. If you need `nvm`, you can get it at
[https://github.com/creationix/nvm](https://github.com/creationix/nvm).
Condenser is known to successfully build using node 8.7, npm 5.4.2, and
yarn 1.3.2.
Install at least Node.js v12 if you don't already have it.
Using nvm, you would install like this:
```bash
nvm install v8.7
```
Condenser is known to successfully build using node 12.6, npm 6.13.4, and
yarn 1.22.4.
We use the yarn package manager instead of the default `npm`. There are
multiple reasons for this, one being that we have `steem-js` built from
source pulling the github repo as part of the build process and yarn
multiple reasons for this, one being that we have `hive-js` built from
source pulling the gitlab repo as part of the build process and yarn
supports this. This way the library that handles keys can be loaded by
commit hash instead of a version name and cryptographically verified to be
exactly what we expect it to be. Yarn can be installed with `npm`, but
......@@ -86,8 +67,8 @@ afterwards you will not need to use `npm` further.
```bash
npm install -g yarn
yarn global add babel-cli
yarn install --frozen-lockfile
yarn add babel-cli
yarn install --frozen-lockfile --ignore-optional
yarn run build
```
To run condenser in production mode, run:
......@@ -109,13 +90,13 @@ yarn run start
It will take quite a bit longer to start in this mode (~60s) as it needs to
build and start the webpack-dev-server.
By default you will be connected to steemit.com's public steem node at
`wss://steemd.steeemit.com`. This is actually on the real blockchain and
By default you will be connected to community public api node at
`https://api.hive.blog`. This is actually on the real blockchain and
you would use your regular account name and credentials to login - there is
not an official separate testnet at this time. If you intend to run a
full-fledged site relying on your own, we recommend looking into running a
copy of `steemd` locally instead
[https://github.com/steemit/steem](https://github.com/steemit/steem).
copy of `hive (steemd)` locally instead
[https://gitlab.syncad.com/hive/hive](https://gitlab.syncad.com/hive/hive).
#### Debugging SSR code
......@@ -134,8 +115,8 @@ stored in `config/defaults.json`.
Environment variables using an example like this:
```bash
export SDC_CLIENT_STEEMD_URL="wss://steemd.steemit.com"
export SDC_SERVER_STEEMD_URL="wss://steemd.steemit.com"
export SDC_CLIENT_STEEMD_URL="https://api.hive.blog"
export SDC_SERVER_STEEMD_URL="https://api.hive.blog"
```
Keep in mind environment variables only exist in your active session, so if
......@@ -217,20 +198,10 @@ OFFLINE_SSR_TEST=true NODE_ENV=production node --prof lib/server/index.js
This will read data from the blobs in `api_mockdata` directory. If you want to use another set of mock data, create a similar directory to that one and add an argument `OFFLINE_SSR_TEST_DATA_DIR` pointing to your new directory.
### Run blackbox tests using nightwatch
To run a Selenium test suite, start the condenser docker image with a name `condenser` (like `docker run --name condenser -itp 8080:8080 steemit/condenser:latest`) and then run the blackboxtest image attached to the condneser image's network:
```
docker build -t=steemit/condenser-blackboxtest blackboxtest/
docker run --network container:condenser steemit/condenser-blackboxtest:latest
```
## Issues
To report a non-critical issue, please file an issue on this GitHub project.
If you find a security issue please report details to: security@steemit.com
If you find a security issue please report details to trusted community members.
We will evaluate the risk and make a patch available before filing the issue.
......@@ -19463,7 +19463,7 @@ module.exports = {
title:
'Why dont we get on BitFinex? They Just Added Loads of Coins!',
body:
'BitFinex has registrations closed even with invites but is one a top volume exchange for BTC.\n\n\nThey started adding loads of coins lately and today they got BAT MANA FUN ZRX TNB which all pumped.\nhttps://i.imgur.com/ErAdfcs.png\n\nhttps://i.imgur.com/WQTnq16.png\n\nWorth to note they have also changed deposit rules due to network and fee problems.\nhttps://i.imgur.com/G4Qancb.png\n\nBinance opened doors today but [you can get only with INVITE](http://fuk.io/binance) (dont thank me) and its open just few times a day. Try to get on when you can.\n\nAs i wrote previously - i suggest to get on [KuCoin](http://KUCOlN.com) now, they will prolly close like others too soon.\n\n## STEEM TO BFX\nThey used to just have BTC then ETH LTC. Now they have loads. I dont see why STEEM cant be there soon if any staff writes a ticket and STEEM is verified by them.\n\nI have tweeted on them to add STEEM, consider to RT for traction.\n\nhttps://i.imgur.com/TrBJldG.png\n\nhttps://twitter.com/kingscrownBTC/status/950545067034513408\n\n<table>\n <tr>\n <th><h1> Follow, Resteem and VOTE UP <a href="/@kingscrown">@kingscrown</a> creator of <a href="http://fuk.io" rel="noopener">http://fuk.io</a> blog for 0day cryptocurrency news and tips!</h1></th>\n <th><img src="https://steemitimages.com/0x0/https://media.giphy.com/media/3ohzdPbKlUFVsmGpq0/giphy.gif" /></th>\n </tr>\n</table>',
'BitFinex has registrations closed even with invites but is one a top volume exchange for BTC.\n\n\nThey started adding loads of coins lately and today they got BAT MANA FUN ZRX TNB which all pumped.\nhttps://i.imgur.com/ErAdfcs.png\n\nhttps://i.imgur.com/WQTnq16.png\n\nWorth to note they have also changed deposit rules due to network and fee problems.\nhttps://i.imgur.com/G4Qancb.png\n\nBinance opened doors today but [you can get only with INVITE](http://fuk.io/binance) (dont thank me) and its open just few times a day. Try to get on when you can.\n\nAs i wrote previously - i suggest to get on [KuCoin](http://KUCOlN.com) now, they will prolly close like others too soon.\n\n## STEEM TO BFX\nThey used to just have BTC then ETH LTC. Now they have loads. I dont see why STEEM cant be there soon if any staff writes a ticket and STEEM is verified by them.\n\nI have tweeted on them to add STEEM, consider to RT for traction.\n\nhttps://i.imgur.com/TrBJldG.png\n\nhttps://twitter.com/kingscrownBTC/status/950545067034513408\n\n<table>\n <tr>\n <th><h1> Follow, Reblog and VOTE UP <a href="/@kingscrown">@kingscrown</a> creator of <a href="http://fuk.io" rel="noopener">http://fuk.io</a> blog for 0day cryptocurrency news and tips!</h1></th>\n <th><img src="https://steemitimages.com/0x0/https://media.giphy.com/media/3ohzdPbKlUFVsmGpq0/giphy.gif" /></th>\n </tr>\n</table>',
json_metadata:
'{"tags":["bitcoin","blockchain","cryptocurrency","steemit","community"],"image":["https://i.imgur.com/ErAdfcs.png","https://i.imgur.com/WQTnq16.png","https://i.imgur.com/G4Qancb.png","https://i.imgur.com/TrBJldG.png","https://steemitimages.com/0x0/https://media.giphy.com/media/3ohzdPbKlUFVsmGpq0/giphy.gif"],"links":["http://fuk.io/binance","http://KUCOlN.com","https://twitter.com/kingscrownBTC/status/950545067034513408","/@kingscrown","http://fuk.io"],"app":"steemit/0.1","format":"markdown"}',
last_update: '2018-01-09T01:51:09',
......
client_max_body_size 16m;
......@@ -25,22 +25,8 @@
"secret_key": "SDC_RECAPTCHA_SECRET_KEY",
"site_key": "SDC_RECAPTCHA_SITE_KEY"
},
"sendgrid": {
"from": "SDC_SENDGRID_FROM",
"key": "SDC_SENDGRID_API_KEY",
"templates": {
}
},
"server_session_secret": "SDC_SESSION_SECRETKEY",
"site_domain": "SDC_SITE_DOMAIN",
"telesign": {
"customer_id": "SDC_TELESIGN_CUSTOMER_ID",
"rest_api_key": "SDC_TELESIGN_API_KEY"
},
"twilio": {
"account_sid": "SDC_TWILIO_ACCOUNT_SID",
"auth_token": "SDC_TWILIO_AUTH_TOKEN"
},
"upload_image": "SDC_UPLOAD_IMAGE_URL",
"session_cookie_key": "SDC_SESSION_COOKIE_KEY",
"steemd_connection_client": "SDC_CLIENT_STEEMD_URL",
......@@ -60,5 +46,6 @@
"steem_market_endpoint": "STEEM_MARKET_ENDPOINT",
"steem_market_token": "STEEM_MARKET_TOKEN",
"cookie_consent_enabled": "SDC_ENABLE_COOKIE_CONSENT",
"cookie_consent_api_key": "SDC_COOKIE_CONSENT_KEY"
"cookie_consent_api_key": "SDC_COOKIE_CONSENT_KEY",
"esteem_elastic_search_api_key": "ESTEEM_ELASTIC_SEARCH_API_KEY"
}
......@@ -4,11 +4,11 @@
"helmet": {
"directives": {
"childSrc": "'self' 3speak.online emb.d.tube player.twitch.tv www.youtube.com staticxx.facebook.com w.soundcloud.com player.vimeo.com",
"connectSrc": "https://steemitimages.com securepubads.g.doubleclick.net 'self' steemit.com https://api.steemit.com api.blocktrades.us",
"connectSrc": "https://images.hive.blog 'self' hive.blog https://api.hive.blog api.blocktrades.us https://anyx.io",
"defaultSrc": "tpc.googlesyndication.com 'self' img.3speakcontent.online emb.d.tube www.youtube.com staticxx.facebook.com player.vimeo.com *.streamrail.com",
"fontSrc": "data: fonts.gstatic.com",
"frameAncestors": "'none'",
"frameSrc": "'self' googleads.g.doubleclick.net https:",
"frameSrc": "'self' https:",
"imgSrc": "* data:",
"objectSrc": "'none'",
"pluginTypes": "application/pdf",
......@@ -19,7 +19,7 @@
"reportOnly": false,
"setAllHeaders": true
},
"img_proxy_prefix": "https://steemitdevimages.com/",
"img_proxy_prefix": "https://images.hive.blog/",
"ipfs_prefix": false,
"mixpanel": false,
"notify": {
......@@ -30,44 +30,32 @@
"secret_key": false,
"site_key": false
},
"sendgrid": {
"from": "noreply@example.com",
"key": "SG.xxx_yyyy",
"templates": {
}
},
"server_session_secret": "exiKdyF+IwRIXJDmtGIl4vWUz4i3eVSISpfZoeYc0s4=",
"session_cookie_key": "stm-dev",
"session_key": "steemses",
"site_domain": "steemitdev.com",
"telesign": {
"customer_id": false,
"rest_api_key": false
},
"twilio": {
"account_sid": false,
"auth_token": false
},
"upload_image": false,
"steemd_connection_client": "https://api.steemit.com",
"steemd_connection_server": "https://api.steemit.com",
"steemd_connection_client": "https://api.hive.blog",
"steemd_connection_server": "https://api.hive.blog",
"steemd_use_appbase": false,
"chain_id": "0000000000000000000000000000000000000000000000000000000000000000",
"alternative_api_endpoints": ["https://api.hive.blog", "https://anyx.io"],
"failover_threshold": 3,
"address_prefix": "STM",
"conveyor_posting_wif": false,
"conveyor_username": false,
"special_posts_url": false,
"wallet_url": "https://wallet.steemitdev.com",
"wallet_url": "https://wallet.hive.blog",
"gpt_banned_tags": ["nsfw", "tante"],
"google_ad_enabled": null,
"google_ad_test": null,
"google_ad_client": null,
"google_ad_slots": {
"market_1": {
"slot_id": "2777377820"
"slot_id": "0"
},
"in_feed_1": {
"slot_id": "9514544128",
"slot_id": "0",
"layout_key": "-fb+5w+4e-db+86",
"every": 15
}
......@@ -238,5 +226,6 @@
"steem_market_cache": {
"ttl": 7200,
"key": "steem-market"
}
},
"esteem_elastic_search_api_key": null
}
#!/usr/bin/env bash
export NODE_ENV=development
echo "Starting Condenser"
yarn start
version: "3"
services:
hive_condenser:
build: .
image: hive/condenser:latest
container_name: hive_condenser
ports:
- 80:8080
- 8080:8080
- 35729:35729
environment:
SDC_CLIENT_STEEMD_URL: https://api.hive.blog
SDC_SERVER_STEEMD_URL: https://api.hive.blog
SDC_DISABLE_SIGNUPS: 1
# add to your /etc/hosts
# 127.0.0.1 hiveblog.local
SDC_SITE_DOMAIN: hiveblog.local
SDC_IMAGE_PROXY_PREFIX: https://images.hive.blog/
SDC_UPLOAD_IMAGE_URL: https://images.hive.blog
SDC_HELMET_CONNECTSRC: "'self' http://hiveblog.local https://api.hive.blog https://images.hive.blog https://anyx.io"
volumes:
- ./yarn.lock:/var/app/yarn.lock
- ./package.json:/var/app/package.json
- ./dev-start.sh:/var/app/dev-start.sh:delegated
- ./dev-ssr.sh:/var/app/dev-ssr.sh:delegated
- ./src:/var/app/src:delegated
command: bash dev-start.sh
version: "3"
services:
hive_condenser:
build: .
image: hive/condenser:latest
container_name: hive_condenser
environment:
VIRTUAL_HOST: hive.blog
VIRTUAL_PORT: 8080
SDC_CLIENT_STEEMD_URL: https://api.hive.blog
SDC_SERVER_STEEMD_URL: https://api.hive.blog
SDC_DISABLE_SIGNUPS: 1
SDC_SITE_DOMAIN: hive.blog
SDC_IMAGE_PROXY_PREFIX: https://steemitimages.com/
SDC_UPLOAD_IMAGE_URL: https://steemitimages.com
SDC_HELMET_CONNECTSRC: "'self' steemit.com https://api.hive.blog api.blocktrades.us https://hive.blog https://steemitimages.com"
WALLET_URL: https://wallet.hive.blog
networks:
- condenser_reverse-proxy
restart: always
# condenser_stg:
# build: .
# image: hive/condenser:staging
# container_name: condenser_stg
# environment:
# VIRTUAL_HOST: staging.hive.blog
# VIRTUAL_PORT: 8080
# SDC_CLIENT_STEEMD_URL: https://api.hive.blog
# SDC_SERVER_STEEMD_URL: https://api.hive.blog
# SDC_DISABLE_SIGNUPS: 1
# SDC_SITE_DOMAIN: staging.hive.blog
# SDC_IMAGE_PROXY_PREFIX: https://steemitimages.com/
# SDC_UPLOAD_IMAGE_URL: https://steemitimages.com
# SDC_HELMET_CONNECTSRC: "'self' https://api.hive.blog https://staging.hive.blog https://steemitimages.com"
# WALLET_URL: https://wallet.hive.blog
# networks:
# - condenser_reverse-proxy
# restart: always
networks:
condenser_reverse-proxy:
external:
name: condenser_reverse-proxy
version: "3"
services:
proxy:
image: jwilder/nginx-proxy:latest
container_name: proxy
labels:
com.github.jrcs.letsencrypt_nginx_proxy_companion.nginx_proxy: "true"
ports:
- "80:80"
- "443:443"
volumes:
- letsencrypt-certs:/etc/nginx/certs:ro
- nginx-vhost:/etc/nginx/vhost.d
- nginx-html:/usr/share/nginx/html
- nginx-conf:/etc/nginx/conf.d
- /var/run/docker.sock:/tmp/docker.sock:ro
- ./client_max_body_size.conf:/etc/nginx/conf.d/client_max_body_size.conf:ro
networks:
- reverse-proxy
restart: always
letsencrypt:
image: jrcs/letsencrypt-nginx-proxy-companion:latest
container_name: letsencrypt
depends_on:
- proxy
environment:
NGINX_PROXY_CONTAINER: proxy
volumes:
- letsencrypt-certs:/etc/nginx/certs
- nginx-vhost:/etc/nginx/vhost.d
- nginx-html:/usr/share/nginx/html
- /var/run/docker.sock:/var/run/docker.sock:ro
networks:
- reverse-proxy
restart: always
networks:
reverse-proxy:
driver: bridge
volumes:
letsencrypt-certs:
driver_opts:
type: none
device: /etc/letsencrypt/certs
o: bind
nginx-conf:
driver_opts:
type: none
device: /etc/nginx/conf.d
o: bind
nginx-vhost:
driver_opts:
type: none
device: /etc/nginx/vhost.d
o: bind
nginx-html:
driver_opts:
type: none
device: /var/www/html
o: bind
version: "3"
services:
hive_condenser_stg:
build: .
image: hive/condenser:staging
container_name: hive_condenser_stg
environment:
VIRTUAL_HOST: staging-blog.hive.io
VIRTUAL_PORT: 8080
LETSENCRYPT_HOST: staging-blog.hive.io
LETSENCRYPT_EMAIL: certs@hive.io
SDC_CLIENT_STEEMD_URL: https://api.hive.blog
SDC_SERVER_STEEMD_URL: https://api.hive.blog
SDC_DISABLE_SIGNUPS: 1
SDC_SITE_DOMAIN: staging-blog.hive.io
SDC_IMAGE_PROXY_PREFIX: https://images.hive.blog/
SDC_UPLOAD_IMAGE_URL: https://images.hive.blog
SDC_HELMET_CONNECTSRC: "'self' https://api.hive.blog https://staging.hive.blog https://images.hive.blog https://anyx.io"
WALLET_URL: https://wallet.hive.blog
networks:
- condenser_reverse-proxy
restart: always
networks:
condenser_reverse-proxy:
external:
name: condenser_reverse-proxy
grep -rin $1 --color=always --exclude-dir=node_modules --exclude-dir=dist --exclude-dir=tmp *
find src | grep -i $1
{
"name": "steemit.com",
"name": "hive.blog",
"repository": {
"type": "git",
"url": "https://github.com/steemit/steemit.com.git"
"url": "https://gitlab.com/hiveblog/tbd.git"
},
"version": "1.0.0",
"description": "steemit.com is the koa web server & middleware and react.js in-browser code for the world's first blockchain content + social media monetization platform!",
......@@ -20,15 +20,16 @@
"storybook": "start-storybook -p 9001 -c .storybook",
"storybook-build": "build-storybook -c .storybook -o docs"
},
"author": "Steemit, Inc.",
"author": "HiveBlog",
"license": "MIT",
"dependencies": {
"@hivechain/hivescript": "^1.1.3",
"@hiveio/hive-js": "^0.0.2",
"@steem/crypto-session": "git+https://github.com/steemit/crypto-session.git#83a90b319ce5bc6a70362d52a15a815de7e729bb",
"@steemit/steem-js": "0.7.5",
"assert": "1.4.1",
"autoprefixer-loader": "3.2.0",
"axios": "^0.18.0",
"babel-cli": "6.26.0",
"babel-cli": "^6.26.0",
"babel-core": "6.26.0",
"babel-eslint": "6.1.2",
"babel-loader": "7.1.2",
......@@ -39,8 +40,7 @@
"babel-preset-es2015": "6.24.1",
"babel-preset-react": "6.24.1",
"babel-preset-stage-0": "6.24.1",
"bigi": "1.4.2",
"blocked": "1.2.1",
"bluebird": "3.5.5",
"blueimp-file-upload": "9.12.6",
"bs58": "3.1.0",
"bytebuffer": "5.0.1",
......@@ -52,15 +52,13 @@
"counterpart": "0.17.9",
"cpu-stat": "2.0.1",
"css-loader": "0.28.7",
"currency-symbol-map": "3.1.0",
"diff-match-patch": "1.0.0",
"diff-match-patch": "1.0.4",
"disk-stat": "1.0.4",
"ecurve": "1.0.5",
"estraverse-fb": "1.3.2",
"file-loader": "0.11.2",
"foundation-sites": "git+https://github.com/steemit/foundation-sites.git#e8e32c715bbc4c822b80b555345f61337269ca78",
"git-rev-sync": "1.9.1",
"highcharts": "4.2.7",
"humanize-number": "0.0.2",
"imports-loader": "0.7.1",
"intl": "1.2.5",
......@@ -89,7 +87,7 @@
"mixpanel": "0.5.0",
"net": "1.0.2",
"node-cache": "^4.2.0",
"node-sass": "4.9.0",
"node-sass": "^4.13.1",
"os": "0.1.1",
"picturefill": "3.0.2",
"qrious": "^4.0.2",
......@@ -97,24 +95,22 @@
"react": "15.6.2",
"react-ab-test": "1.7.0",
"react-addons-pure-render-mixin": "15.4.2",
"react-autocomplete": "1.7.2",
"react-autocomplete": "1.8.1",
"react-copy-to-clipboard": "4.3.1",
"react-dom": "15.6.2",
"react-dropzone": "3.13.4",