Mischiefblog
I make apps for other people

General

Resizing or shrinking KVM virtual machine filesystems and disk images

Posted by Chris Jones
On September 16th, 2014 at 11:22

Permalink | Trackback | Links In |

Comments Off
Posted in General

Most everyone wants to increase the size of their disk images, filesystems, or virtual machines (VM). It’s common to create a small, minimal machine (2 GB in size) and increase it depending on the flavor or configuration of the deployed VM.

However, there are some cases where machines will need to be moved from large disk flavors to smaller flavors. Assuming the filesystem isn’t full or is minimal in size, you may be able to successfully resize the disk image so that it will fit on a smaller VM.
(more…)

Example cgo (Golang) app that calls a native library with a C structure

Posted by Chris Jones
On June 26th, 2014 at 12:47

Permalink | Trackback | Links In |

Comments Off
Posted in General

Here’s a sample Go application that calls a native library. This uses cgo to link to the native library, pass a pointer to a C structure to a native function, and does it all from Go.

This example includes:

  • a Go main
  • a Go package
  • a static C library (shared libraries on OS X require tweaks to dyld)
  • instructions for compilation

(more…)

A Go (cgo) gotcha

Posted by Chris Jones
On June 24th, 2014 at 16:24

Permalink | Trackback | Links In |

Comments Off
Posted in General

If you get a lot of errors like:

37: error: use of undeclared identifier 'create_message'

in your Go interface to a C library, double check your imports. You can’t use imports in the style:

/*
* #cgo CFLAGS: -I../libdir
* #cgo LDFLAGS: -L../libdir -lmylib
* #include "mylib.h"
*/
import (
"C"
"unsafe"
)

Instead, use each import on a separate line, or make the first import “C” so that cgo will pick up the header include:

/*
* #cgo CFLAGS: -I../libdir
* #cgo LDFLAGS: -L../libdir -lmylib
* #include "mylib.h"
*/
import "C"
import "unsafe"

Also, make sure that if you’re using a local shared library (.so) that you symlink it to a .so name:

$ gcc -c -fPIC mylib.c -o mylib.o
$ gcc -shared -W1,-soname,libmylib.so.1 -o libmylib.so.1.0.0 mylib.o
$ ln -s libmylib.so.1.0.0 libmylib.so

A simple FastCGI bottle.py Python web application for Dreamhost

Posted by Chris Jones
On August 18th, 2012 at 16:55

Permalink | Trackback | Links In |

Comments Off
Posted in General

Here are instructions for setting up a simple bottle.py Python web application running on Dreamhost (shared) with virtualenv.
(more…)

HOWTO Create a CSV report from an Oracle query

Posted by Chris Jones
On August 3rd, 2012 at 13:09

Permalink | Trackback | Links In |

Comments Off
Posted in General

Here’s a simple shell script that turns an Oracle query into a CSV file suitable for mailing.
(more…)

HOWTO change your password in Oracle

Posted by Chris Jones
On July 25th, 2012 at 16:35

Permalink | Trackback | Links In |

Comments Off
Posted in General

With SQLPlus

Log into Oracle with SQLPlus.$ sqlplus user@dbname
SQL*Plus: Release 11.1.0.6.0 - Production on Mon Apr 12 11:46:48 2010

Copyright (c) 1982, 2007, Oracle. All rights reserved.

Enter password: <enter it here>

Connected to:
Oracle Database 11g Enterprise Edition Release 11.2.0.1.0 - 64bit Production
With the Partitioning, Real Application Clusters, Automatic Storage Management, OLAP,
Data Mining and Real Application Testing options

SQL> password
Changing password for USER
Old password: <old pass>
New password: <new pass>
Retype new password: <new pass>
Password changed
SQL> CTRL-D
Disconnected from Oracle Database 11g Enterprise Edition Release 11.2.0.1.0 - 64bit Production
With the Partitioning, Real Application Clusters, Automatic Storage Management, OLAP,
Data Mining and Real Application Testing options

With SQL Developer
To change your password in SQL Developer, log into the database in SQL Developer and execute the following statement:

alter user username identified by "newpassword" replace "oldpassword"

The quotes make the password safe for punctuation characters.

Shell scripts that make my life easier

Posted by Chris Jones
On July 25th, 2012 at 15:53

Permalink | Trackback | Links In |

Comments Off
Posted in General

Below, a series of shell scripts that make my life easier. Don’t expect any great revelations or virtuoso bash scripting.
(more…)

HOWTO proxy with nginx

Posted by Chris Jones
On July 25th, 2012 at 13:51

Permalink | Trackback | Links In |

Comments Off
Posted in General

These instructions assume you’re using a Debian/Ubuntu-based system.

  1. Create a web service/app (i.e., a Maven CXF JAX-RS archetype, or Flask/Bottle/Django, etc.)
  2. Install curl
    sudo apt-get install curl
  3. Install nginx
    sudo apt-get install nginx-full
    which should include nginx-common as a dependency.
  4. Do not start nginx yet.
  5. Add the nginx load balancer configuration and mime.types to your web service project (called MyProject in the following examples)
    1. Copy the mime.types definition file (as an alternative to copying the nginx configuration to /etc/nginx)
      cp /etc/nginx/mime.type ~/workspace/MyProject/web/conf
    2. Copy and modify the following load balancer/proxy configuration into MyProject/web/conf/balancer.nginx

      worker_processes 1;
          
      events {
          worker_connections 64;
          # multi_accept on;
      }
          
      http {
          upstream service-backend {
              server 127.0.0.1:9260;
              # server 127.0.0.1:8102;
              # server 127.0.0.1:8103;
              # server 127.0.0.1:8104;
          }
          
          keepalive_timeout 300 300;
          charset utf-8;
          default_type application/octet-stream;
          ignore_invalid_headers on;
          include mime.types;
          keepalive_requests 20;
          recursive_error_pages on;
          sendfile on;
          server_tokens off;
          source_charset utf-8;
          gzip on;
          gzip_static on;
          
          log_format main '$remote_addr $host $remote_user [$time_local] "$request" $status $body_bytes_sent "$http_referer" "$http_user_agent" $ssl_cipher $request_time';
          
          server {
              listen 127.0.0.1:8100;
              server_name mydesktop.domain.com mydesktop;
              add_header Cache-Control public;
              access_log /var/log/nginx/access.log main buffer=32k;
              error_log /var/log/nginx/error.log error buffer=8k;
              expires max;
              root /opt/MyProject/web/html;
          
              location /svc/ {
                  proxy_pass http://127.0.0.1:9260;
              }
          
              location / {
              
              }
          
              location /favicon.ico {
                  return 204;
              }
              }
      }
  6. Create a directory to hold your static web content (HTML pages, images, JavaScript)
    mkdir MyProject/web/html
  7. Create a placeholder HTML file in the static web content/html directory
  8. Set your webservice to run on the /svc URI endpoint (to match the nginx configuration)
    1. Update the MyProject integration tests

      <parameter name="com.domain.webservices.AppRest.url"
      type="text" desc="Example service URL">
      %<value>http://localhost:9260/MyProject/svc</value>
      </parameter>
    2. Update the MyProject/web/conf/web.xml to map the Jersey web application to “/svc/*” (or whatever is appropriate for your app)
      <servlet-mapping>
      <servlet-name>Jersey Web Application/svc/*
  9. Build your project
    cd ~/workspace/MyProject
    mvn clean install
  10. Install and start your app (under /opt, for instance)
  11. Start nginx as root
    nginx -c /opt/Myproject/web/conf/balancer.nginx
  12. Verify your deployment
    1. Check your placeholder HTML page
      curl http://localhost:8100/index.html
    2. Check your service health
      curl http://localhost:8100/svc/health

HOWTO minify JavaScript

Posted by Chris Jones
On July 25th, 2012 at 13:32

Permalink | Trackback | Links In |

Comments Off
Posted in General
  1. Get a copy of jsmin.c from http://crockford.com/javascript/jsmin
  2. Compile jsmin with
    gcc -o jsmin jsmin.c
  3. Put jsmin in your path (i.e., under ~/bin)
  4. Minify JavaScript
  5. jsmin < input.js > output.min.js

As a best practice, when modifying third-party code rename your JavaScript file to something like filename.domain.min.js.

Scalability Secrets: custom content that scales

Posted by Chris Jones
On July 15th, 2012 at 11:35

Permalink | Trackback | Links In |

Comments Off
Posted in General

Caveat: this isn’t about failover, security, or cloud computing.

Imagine you have 65 million registered users and you need to provide custom content for each (beyond “Hello, Bob”), say real time subscription content from tens of thousands of sources containing millions of posts.

Don’t try to show everything at once

You probably won’t be able to show everything to the user at once. Depending on your load and back-end systems, you may be lucky to simply let the registered user know that they have something to see.

Most of your users won’t be logged in

Most page views (at least to a homepage) won’t be on logged-in users: many will be first time users, or users who haven’t visited very often. You’ll need to hit your back-end servers with a specific ratio of total visits, something you can measure early, and use as a baseline for how many servers you need to scale.
(more…)