OpenShift, Fedora,

and the future of packaged web apps

Created by Andy Grimm / @a13m

3 Free Gears!

Operating System... versus Distribution

What's the point of packaging a distribution?


  • Quality
  • Security
  • "Distributability" (legal, technical, and otherwise)
  • User Experience
  • Admin Experience
  • Developer Experience
    •

Proliferation


of languages...

  • Perl
  • Python
  • PHP
  • Ruby
  • Java
  • NodeJS
    •
and dependency / package managers...

  • => CPAN
  • => pip
  • => PECL / PEAR
  • => gem
  • => Maven
  • => npm
 		and environment managers

  • =>
  • => virtualenv
  • =>
  • => Bundler / rbenv
  • =>
  • =>

On use cases...


  • Single-user
  • versus Multi-user
  • versus Multi-tenant

Ok, so not quite a Pale Blue Dot








OSS Ecosystem

A quick poll of the audience

Why PaaS (instead of IaaS)?

  • Real people don't build images
  • nor do they care to deal with upgrades
  • and it's saves them money

Is OpenShift PaaS? YES

Is it FaaS? YES

What is a "gear", anyway?

As of today...

  • a "locked" home directory
  • namespaced mounts
  • cgroup'ed processes
  • a restricted shell
  • one or more "cartridges"

What is restricted?

  • Disk, cpu, memory, IP/port binding, nproc, nofile, etc.
  • Using quota, cgroup, ulimit, pam, SELinux, iptables
  • Process table visibility is restricted, but netstat is not

Show me an example, already!

Example: home directory


[wp-agrimm.rhcloud.com 51f0013a4382eccdc5002bc0]\> ls -la
total 60
drwxr-x---.  11 root     51f0013a  4096 Jul 24 12:31 .
drwxr-x--x. 202 root     root     12288 Aug  2 12:48 ..
drwxr-xr-x.   4 root     51f0013a  4096 Jul 24 12:30 app-root
drwxr-x---.   2 root     51f0013a  4096 Jul 24 12:31 .env
drwxr-x---.   2 51f0013a 51f0013a  4096 Jul 24 12:30 .gem
drwxr-xr-x.   3 root     root      4096 Jul 24 12:31 git
-rw-r--r--.   1 root     root        57 Jul 24 12:31 .gitconfig
drwxr-xr-x.  12 51f0013a 51f0013a  4096 Jul 24 12:31 mysql
-rw-r--r--.   1 root     51f0013a  1222 Jul 24 12:31 .pearrc
drwxr-xr-x.  14 51f0013a 51f0013a  4096 Jul 24 12:31 php
d---------.   3 root     root      4096 Jul 24 12:30 .sandbox
drwxr-x---.   2 root     51f0013a  4096 Jul 24 12:31 .ssh
d---------.   3 root     root      4096 Jul 24 12:31 .tmp

Example: environment


[wp-agrimm.rhcloud.com 51f0013a4382eccdc5002bc0]\> env | grep OPENSHIFT | sort
OPENSHIFT_APP_DNS=wp-agrimm.rhcloud.com
OPENSHIFT_APP_NAME=wp
OPENSHIFT_APP_UUID=51f0013a4382eccdc5002bc0
OPENSHIFT_BROKER_HOST=openshift.redhat.com
OPENSHIFT_CARTRIDGE_SDK_BASH=/usr/lib/openshift/cartridge_sdk/bash/sdk
OPENSHIFT_CARTRIDGE_SDK_RUBY=/usr/lib/openshift/cartridge_sdk/ruby/sdk.rb
OPENSHIFT_CLOUD_DOMAIN=rhcloud.com
OPENSHIFT_DATA_DIR=/var/lib/openshift/51f0013a4382eccdc5002bc0/app-root/data/
OPENSHIFT_GEAR_DNS=wp-agrimm.rhcloud.com
OPENSHIFT_GEAR_NAME=wp
OPENSHIFT_GEAR_UUID=51f0013a4382eccdc5002bc0
OPENSHIFT_HOMEDIR=/var/lib/openshift/51f0013a4382eccdc5002bc0/
OPENSHIFT_MYSQL_DB_HOST=127.9.120.2
OPENSHIFT_MYSQL_DB_LOG_DIR=/var/lib/openshift/51f0013a4382eccdc5002bc0/mysql//log/
OPENSHIFT_MYSQL_DB_PASSWORD=KBQTSSHnswHx
OPENSHIFT_MYSQL_DB_PORT=3306
OPENSHIFT_MYSQL_DB_SOCKET=/var/lib/openshift/51f0013a4382eccdc5002bc0/mysql//socket/mysql.sock
OPENSHIFT_MYSQL_DB_URL=mysql://adminw4AIWMf:KBQTSSHnswHx@127.9.120.2:3306/
OPENSHIFT_MYSQL_DB_USERNAME=adminw4AIWMf
OPENSHIFT_MYSQL_DIR=/var/lib/openshift/51f0013a4382eccdc5002bc0/mysql/
OPENSHIFT_MYSQL_IDENT=redhat:mysql:5.1:0.2.0
OPENSHIFT_NAMESPACE=agrimm
OPENSHIFT_PHP_DIR=/var/lib/openshift/51f0013a4382eccdc5002bc0/php/
OPENSHIFT_PHP_IDENT=redhat:php:5.3:0.0.2
OPENSHIFT_PHP_IP=127.9.120.1
OPENSHIFT_PHP_LOG_DIR=/var/lib/openshift/51f0013a4382eccdc5002bc0/php//logs/
OPENSHIFT_PHP_PORT=8080
OPENSHIFT_PHP_VERSION=5.3
OPENSHIFT_PRIMARY_CARTRIDGE_DIR=/var/lib/openshift/51f0013a4382eccdc5002bc0/php/
OPENSHIFT_REPO_DIR=/var/lib/openshift/51f0013a4382eccdc5002bc0/app-root/runtime/repo/
OPENSHIFT_TMP_DIR=/tmp/

Example: Process tree


[wp-agrimm.rhcloud.com 51f0013a4382eccdc5002bc0]\> ps auxww
USER       PID %CPU %MEM    VSZ   RSS TTY      STAT START   TIME COMMAND
4848     17633  0.0  0.0 108104   904 pts/0    R+   13:14   0:00 ps auxww
4848     27368  0.0  0.0 102432  1872 ?        S    13:01   0:00 sshd: 51f0013a4382eccdc5002bc0@pts/0
4848     27369  0.0  0.0 108600  2064 pts/0    Ss   13:01   0:00 /bin/bash --init-file /usr/bin/rhcsh -i

Example: Port binding


[wp-agrimm.rhcloud.com 51f0013a4382eccdc5002bc0]\> cat <<EOF | python -
from os import environ; from socket import socket
def try_bind(x):
    s = socket()
    try:
       print "Trying to bind to %s:%s" % x ; s.bind(x)
       print "success"                                
    except Exception, e:
       print "failed: %s" % e
                             
try_bind(("127.0.0.1", 8000))
try_bind((environ['OPENSHIFT_PHP_IP'], 8000))
try_bind((environ['OPENSHIFT_PHP_IP'], int(environ['OPENSHIFT_PHP_PORT'])))
EOF                                                                        

Trying to bind to 127.0.0.1:8000
failed: [Errno 13] Permission denied
Trying to bind to 127.9.120.1:8000
failed: [Errno 13] Permission denied
Trying to bind to 127.9.120.1:8080
success

How to build an app inside a gear

  1. Install at least one cartridge which "publishes" a port
  2. Install dependencies
  3. Write code/config to determine valid IP/ports from environment
  4. Configure the application for non-privileged use

What is a cartridge?


A manifest, a set of control scripts, and supporting files to provide one service

What about quickstarts?


  • It's just code for running an app on top cartridges
  • Users clone into their git repo
  • No automated maintenance; just git merges

The PHP Cartridge Manifest


Name: php
Cartridge-Short-Name: PHP
Display-Name: PHP 5.3
Description: "PHP is a general-purpose server-side scripting language originally designed for Web development to produce dynamic Web pages. Popular development frameworks include: CakePHP, Zend, Symfony, and Code Igniter."
Version: '5.3'
License: "The PHP License, version 3.0"
License-Url: http://www.php.net/license/3_0.txt
Vendor: php.net
Cartridge-Version: 0.0.3
Compatible-Versions: [0.0.2]
Cartridge-Vendor: redhat
Categories:
  - service
  - php
  - web_framework
Website: http://www.php.net
Help-Topics:
  "Developer Center": https://www.openshift.com/developers
Cart-Data:
  - Key: OPENSHIFT_TMP_DIR
    Type: environment
    Description: "Directory to store application temporary files."
  - Key: OPENSHIFT_REPO_DIR
    Type: environment
    Description: "Application root directory where application files reside. This directory is reset every time you do a git-push"
  - Key: OPENSHIFT_PHP_PORT
    Type: environment
    Description: "Internal port to which the web-framework binds to."
  - Key: OPENSHIFT_PHP_IP
    Type: environment
    Description: "Internal IP to which the web-framework binds to."
  - Key: OPENSHIFT_APP_DNS
    Type: environment
    Description: "Fully qualified domain name for the application."
  - Key: OPENSHIFT_APP_NAME
    Type: environment
    Description: "Application name"
  - Key: OPENSHIFT_DATA_DIR
    Type: environment
    Description: "Directory to store application data files. Preserved across git-pushes. Not shared across gears."
  - Key: OPENSHIFT_APP_UUID
    Type: environment
    Description: "Unique ID which identified the application. Does not change between gears."
  - Key: OPENSHIFT_GEAR_UUID
    Type: environment
    Description: "Unique ID which identified the gear. This value changes between gears."
Provides:
  - php-5.3
  - "php"
Publishes:
  get-php-ini:
    Type: "FILESYSTEM:php-ini"
  publish-http-url:
    Type: "NET_TCP:httpd-proxy-info"
  publish-gear-endpoint:
    Type: "NET_TCP:gear-endpoint-info"
Subscribes:
  set-env:
    Type: "ENV:*"
    Required: false
  set-mysql-connection-info:
    Type: "NET_TCP:db:mysql"
    Required : false
  set-postgres-connection-info:
    Type: "NET_TCP:db:postgres"
    Required : false
  set-doc-url:
    Type: "STRING:urlpath"
    Required : false
Scaling:
  Min: 1
  Max: -1

Endpoints:
  - Private-IP-Name:   IP
    Private-Port-Name: PORT
    Private-Port:      8080
    Public-Port-Name:  PROXY_PORT
    Mappings:
      - Frontend:      ""
        Backend:       ""
        Options:       { websocket: true }
      - Frontend:      "/health"
        Backend:       ""
        Options:       { health: true }

The PHP Cartridge Layout

  • bin/{setup,teardown,install,control}
  • usr/template/php/{index.php,healthcheck.php}
  • versions/shared/configuration/etc/
    • conf.d/php.conf, conf.d/openshift.conf.erb
    • php.ini.erb
    • conf/httpd_nolog.conf
  • hooks/publish-http-url
  • env/OPENSHIFT_PHP_LOG_DIR.erb

The PHP httpd openshift.conf


ServerRoot "<%= ENV['OPENSHIFT_PHP_DIR'] %>"
DocumentRoot "<%= "#{ENV['OPENSHIFT_REPO_DIR']}/php" %>"
Listen <%= "#{ENV['OPENSHIFT_PHP_IP']}:#{ENV['OPENSHIFT_PHP_PORT']}" %>
User  <%= ENV['OPENSHIFT_GEAR_UUID'] %>
Group <%= ENV['OPENSHIFT_GEAR_UUID'] %>
ErrorLog "<%= "|/usr/sbin/rotatelogs #{ENV['OPENSHIFT_PHP_DIR']}/logs/error_log-%Y%m%d-%H%M%S-%Z 86400" %>"
CustomLog "<%= "|/usr/sbin/rotatelogs #{ENV['OPENSHIFT_PHP_DIR']}/logs/access_log-%Y%m%d-%H%M%S-%Z 86400" %>" combined
= 2.4>
DefaultRuntimeDir "<%= "#{ENV['OPENSHIFT_PHP_DIR']}/run"%>"

php_value include_path "<%= ".:#{ENV['OPENSHIFT_REPO_DIR']}/libs/:#{ENV['OPENSHIFT_PHP_DIR']}/phplib/pear/pear/php/:/usr/share/pear/"%>"

">
  AllowOverride All

PHP dep solving (bin/control build)


for f in $(cat ${OPENSHIFT_REPO_DIR}deplist.txt)
do
    echo "Checking pear: $f"
    echo
    if pear list "$f" > /dev/null
    then
        pear upgrade "$f"
    elif ! ( php -c "${OPENSHIFT_PHP_DIR}"/conf -m | grep -i -q \^`basename "$f"`\$ )
    then
        pear install --alldeps "$f"
    else
        echo "Extension already installed in the system: $f"
    fi
done

Wordpress as a quickstart

  • upstream WP
  • wp-config.php
  • "deploy" hook

Wordpress as a cartridge

  • copy the php cart and s/_PHP_/_WORDPRESS_/
  • Symlinks to /usr/share/wordpress were EPIC FAIL due to ABSPATH
  • copying under cart's usr/ has the same problem
  • so each gear gets its own copy
  • ... but the system can manage updates!

Problems

  • Multi-versions implies _all_ versions must be on the system
  • Cartridges can do _anything_ the gear can do, and more
  • ...including take over your account
  • App upgrades must be separate from RPM upgrades
  • Inter-cartridge deps (Configure-Order et al.)

Trivial cart RPM packaging? or none?

Possible problems with Fedora web apps today

  • Security contexts / isolation
  • No separation of control scripts
  • Lack of non-root administration
  • Port conflicts for non-apache apps

Some guidelines

  • Bundling _might_ be okay
  • Downloading from a static URL is likely not
  • Apps need to deal with alternate config paths
  • control scripts should work outside OpenShift
  • cartridge development community in github
  • would be nice if the same carts work in RHEL+EPEL
  • What's this developer's assistant thing I just heard about?

Questions

BY Andy Grimm / @a13m