news

2005 2006 2007 2008 2009 2010 2011 2012 2013 2014 2015 2016 2017 FOSS devops documentation emacs fedora foss freedom gnome haskell install laptop lisp photo ruby travel verilog vhdl vlsi xmonad


[Published in Open Source For You (OSFY) magazine, April 2017 edition.]

This is the second article in the DevOps series, and covers the installation of a (G/)LAMP stack and WordPress, using Ansible.

Introduction

In this article, we are going to learn how to automate the deployment of a (G/)LAMP stack and install WordPress. (G/)LAMP stands for GNU/Linux, Apache (a web server), MySQL (a database) and PHP (server-side scripting). It is a technology stack on which you can deploy different web applications. We are also going to explore the installation of WordPress, which is free and open source software for creating websites and blogs.

Linux

A Parabola GNU/Linux-libre x86_64 system is used as the host system. An Ubuntu 15.04 image runs as a guest OS using KVM/QEMU. Ansible is installed on the host system using the distribution package manager. You should be able to issue commands from Ansible to the guest OS. For example:

$ ansible ubuntu -m ping

ubuntu | SUCCESS => {
    "changed": false, 
    "ping": "pong"
}

The /etc/hosts file already has an entry for the guest Ubuntu VM.

192.168.122.250 ubuntu

On the host system, we will create a project for our playbooks. It has the following directory structure:

ansible/inventory/kvm/
       /playbooks/configuration/
       /playbooks/admin/

An ‘inventory’ file is created inside the inventory/kvm folder that contains the following:

ubuntu ansible_host=192.168.122.250 ansible_connection=ssh ansible_user=xetex

Installing Apache

We will first install and test the Apache web server on the guest Ubuntu system. Let’s then create a playbooks/configuration/apache.yml file with the following content:

---
- name: Install Apache web server
  hosts: ubuntu
  become: yes
  become_method: sudo
  gather_facts: true
  tags: [web]

  tasks:
    - name: Update the software package repository
      apt:
	update_cache: yes

    - name: Install Apache
      package:
	name: "{{ item }}"
	state: latest
      with_items:
	- apache2

    - wait_for:
	port: 80

On the Ubuntu guest system, the playbook runs apt-get update and then installs the apache2 package. The playbook finishes after the server has started, and is listening on port 80. Open a terminal, enter the ansible/ folder, and execute the playbook as shown below:

$ ansible-playbook -i inventory/kvm/inventory playbooks/configuration/apache.yml -K 
SUDO password: 

PLAY [Install Apache web server] ***********************************************

TASK [setup] *******************************************************************
ok: [ubuntu]

TASK [Update the software package repository] **********************************
changed: [ubuntu]

TASK [Install Apache] **********************************************************
changed: [ubuntu] => (item=[u'apache2'])

TASK [wait_for] ****************************************************************
ok: [ubuntu]

PLAY RECAP *********************************************************************
ubuntu                     : ok=4    changed=2    unreachable=0    failed=0   

The ’-K’ option is to prompt for the sudo password for the ‘xetex’ user. You can increase the level of verbosity in the Ansible output by passing ’-vvvv’ at the end of the ansible-playbook command. The more number of times ‘v’ occurs, the greater is the verbosity level.

If you now open http://192.168.122.250, you should be able to see the default Apache2 index.html page as shown in Figure 1:

Apache2 Ubuntu default page

Installing MySQL

The next step is to install the MySQL database server. The corresponding playbook is provided below:

---
- name: Install MySQL database server
  hosts: ubuntu
  become: yes
  become_method: sudo
  gather_facts: true
  tags: [database]

  tasks:
    - name: Update the software package repository
      apt:
	update_cache: yes

    - name: Install MySQL
      package:
	name: "{{ item }}"
	state: latest
      with_items:
	- mysql-server
	- mysql-client
	- python-mysqldb

    - name: Start the server
      service:
	name: mysql
	state: started

    - wait_for:
	port: 3306

    - mysql_user:
	name: guest
	password: '*F7B659FE10CA9FAC576D358A16CC1BC646762FB2'
	encrypted: yes
	priv: '*.*:ALL,GRANT'
	state: present

The package repository is updated and the necessary MySQL packages are installed. The database server is then started, and we wait for the server to be up and running. A ‘guest’ user account with ‘osfy’ as the password is created for use in our web application. The chosen password is just for demonstration purposes. When used in production, please select a strong password with special characters and numerals.

You can compute the hash for a password from the MySQL client, as shown below:

mysql> SELECT PASSWORD('osfy');
+-------------------------------------------+
| PASSWORD('osfy')                          |
+-------------------------------------------+
| *F7B659FE10CA9FAC576D358A16CC1BC646762FB2 |
+-------------------------------------------+
1 row in set (0.00 sec)

An execution run to install MySQL is as follows:

$ ansible-playbook -i inventory/kvm/inventory playbooks/configuration/mysql.yml -K 
SUDO password: 

PLAY [Install MySQL database server] *******************************************

TASK [setup] *******************************************************************
ok: [ubuntu]

TASK [Update the software package repository] **********************************
changed: [ubuntu]

TASK [Install MySQL] ***********************************************************
changed: [ubuntu] => (item=[u'mysql-server', u'mysql-client', u'python-mysqldb'])

TASK [Start the server] ********************************************************
ok: [ubuntu]

TASK [wait_for] ****************************************************************
ok: [ubuntu]

TASK [mysql_user] **************************************************************
ok: [ubuntu]

PLAY RECAP *********************************************************************
ubuntu                     : ok=6    changed=2    unreachable=0    failed=0   

Note: The default MySQL root password is empty. You should change it after installation.

Installing PHP

PHP is a server-side programming language and stands for PHP: Hypertext Preprocessor (a recursive acronym). Although we have used PHP5 in ths example, it is recommended that you use the latest PHP for security reasons. The Ansible playbook for installing PHP is given below:

---
- name: Install PHP
  hosts: ubuntu
  become: yes
  become_method: sudo
  gather_facts: true
  tags: [web]

  tasks:
    - name: Update the software package repository
      apt:
	update_cache: yes

    - name: Install PHP
      package:
	name: "{{ item }}"
	state: latest
      with_items:
	- php5
	- php5-mysql

We update the software repository and install PHP5. An execution output of the Ansible playbook is shown below:

$  ansible-playbook -i inventory/kvm/inventory playbooks/configuration/php.yml -K 
SUDO password: 

PLAY [Install PHP] *************************************************************

TASK [setup] *******************************************************************
ok: [ubuntu]

TASK [Update the software package repository] **********************************
changed: [ubuntu]

TASK [Install PHP] *************************************************************
changed: [ubuntu] => (item=[u'php5', u'php5-mysql'])

PLAY RECAP *********************************************************************
ubuntu                     : ok=3    changed=2    unreachable=0    failed=0  

Installing WordPress

As a final step, we will download, install and set up WordPress. The complete playbook is as follows:

---
- name: Setup Wordpress
  hosts: ubuntu
  become: yes
  become_method: sudo
  gather_facts: true
  tags: [database]

  vars:
    wordpress_file: "/home/{{ ansible_user }}/Downloads/wordpress-latest.zip"
    wordpress_dest: "/var/www/html"

  tasks:
    - name: Update the software package repository
      apt:
	update_cache: yes

    - name: Create a database for wordpress
      mysql_db:
	name: wordpress
	state: present

    - name: Create downloads directory
      file:
	path: "/home/{{ ansible_user }}/Downloads"
	state: directory

    - name: Create target directory
      file:
	path: "{{ wordpress_dest }}/wordpress"
	state: directory

    - name: Download latest wordpress
      get_url:
	url: https://wordpress.org/latest.zip
	dest: "{{ wordpress_file }}"

    - name: Extract to /var/www/html
      unarchive:
	src: "{{ wordpress_file }}"
	dest: "{{ wordpress_dest}}"
	remote_src: True

    - name: Copy wp-config-sample.php to wp-config.php
      command: cp "{{ wordpress_dest }}/wordpress/wp-config-sample.php" "{{ wordpress_dest }}/wordpress/wp-config.php"

    - name: Update database credentials in the file
      replace:
	dest: "{{ wordpress_dest }}/wordpress/wp-config.php"
	regexp: "{{ item.regexp }}"
	replace: "{{ item.replace }}"
      with_items:
	- { regexp: 'database_name_here', replace: 'wordpress' }
	- { regexp: 'username_here', replace: 'guest' }
	- { regexp: 'password_here', replace: 'osfy'}

    - name: Restart apache2 server
      service:
	name: apache2
	state: restarted

We create variables to store the downloaded file for WordPress, and the target installation path. After updating the software repository, a database is created for the WordPress application. The download and target directories are created on the guest system, before actually downloading the latest WordPress sources. A configuration file is then created, and the database settings are updated. Although we explicitly specify the password here, the recommended practice is to store the encrypted passwords in an Ansible Vault file, and reference the same in the playbook. In future articles, I will demonstrate this use case. After completing the configuration, the web server is restarted. An execution run of the playbook is shown below:

 $  ansible-playbook -i inventory/kvm/inventory playbooks/configuration/wordpress.yml -K 
SUDO password: 

PLAY [Setup Wordpress] *********************************************************

TASK [setup] *******************************************************************
ok: [ubuntu]

TASK [Update the software package repository] **********************************
changed: [ubuntu]

TASK [Create a database for wordpress] *****************************************
changed: [ubuntu]

TASK [Create downloads directory] **********************************************
ok: [ubuntu]

TASK [Create target directory] *************************************************
changed: [ubuntu]

TASK [Download latest wordpress] ***********************************************
ok: [ubuntu]

TASK [Extract to /var/www/html] ************************************************
changed: [ubuntu]

TASK [Copy wp-config-sample.php to wp-config.php] ******************************
changed: [ubuntu]

TASK [Update database credentials in the file] *********************************
changed: [ubuntu] => (item={u'regexp': u'database_name_here', u'replace': u'wordpress'})
changed: [ubuntu] => (item={u'regexp': u'username_here', u'replace': u'guest'})
changed: [ubuntu] => (item={u'regexp': u'password_here', u'replace': u'osfy'})

TASK [Restart apache2 server] **************************************************
changed: [ubuntu]

PLAY RECAP *********************************************************************
ubuntu                     : ok=10   changed=7    unreachable=0    failed=0   

If you open the URL http://192.168.122.250/wordpress in a browser on the host system, you will see a screenshot as shown in Figure 2:

WordPress default page

You can now proceed to complete the installation process from the browser. It is recommended that you follow the security best practices as recommended by the WordPress and PHP projects to harden this deployment.

Writing clean-up playbooks

It is essential to write clean-up playbooks to revert whatever changes you have made, so that you can roll back the system if things fail. Uninstalling should be done in the reverse order. For example, remove WordPress first, followed by PHP, MySQL and Apache.

The removal of WordPress could depend on your data retention policy. You might want to backup your PHP files, or you may decide to discard them. You might also want to retain the database. A complete removal of WordPress and the (G/)LAMP stack in the playbooks/admin folder is provided below for reference:

---
- name: Uninstall Wordpress
  hosts: ubuntu
  become: yes
  become_method: sudo
  gather_facts: true
  tags: [web]

  vars:
    wordpress_dest: "/var/www/html"

  tasks:
    - name: Delete wordpress folder
      file:
	path: "{{ wordpress_dest }}/wordpress"
	state: absent

    - name: Drop database
      mysql_db:
	name: wordpress
	state: absent

---
- name: Uninstall PHP
  hosts: ubuntu
  become: yes
  become_method: sudo
  gather_facts: true
  tags: [web]

  tasks:
    - name: Uninstall PHP packages
      package:
	name: "{{ item }}"
	state: absent
      with_items:
	- php5-mysql
	- php5

---
- name: Uninstall MySQL
  hosts: ubuntu
  become: yes
  become_method: sudo
  gather_facts: true
  tags: [database]

  tasks:
    - name: Stop the database server
      service:
	name: mysql
	state: stopped

    - name: Uninstall MySQL packages
      package:
	name: "{{ item }}"
	state: absent
      with_items:
	- python-mysqldb
	- mysql-client
	- mysql-server

---
- name: Uninstall Apache web server
  hosts: ubuntu
  become: yes
  become_method: sudo
  gather_facts: true
  tags: [server]

  tasks:
    - name: Stop the web server
      service:
	name: apache2
	state: stopped

    - name: Uninstall apache2
      package:
	name: "{{ item }}"
	state: absent
      with_items:
	- apache2

The entire suite of playbooks is also available in my GitHub project ( https://github.com/shakthimaan/introduction-to-ansible ) for your reference.

[Published in Open Source For You (OSFY) magazine, March 2017 edition.]

With this article, we begin a new series on DevOps, starting out with Ansible, which helps you to build a strong foundation. As the Ansible website proclaims, proudly, “Deploy apps. Manage systems. Crush complexity.”

Introduction

Ansible is an IT automation tool that is used for provisioning, configuration, deployment and managing infrastructure. The project was first released in 2012, and is written in Python. The main objective of the tool is to be simple and easy to use. It is based on an agent-less (push-based) architecture, and the playbooks are written in plain English. Today, it also supports pull-based deployments. Ansible uses SSH to execute commands on remote machines. It is available under the GNU General Public License.

Installation

You can install Ansible using your GNU/Linux distribution package manager.

On Fedora, you can use Yum to install Ansible, as follows:

$ sudo yum install ansible

If you are using RHEL or CentOS, install the epel-release, and then use the Yum command to install Ansible.

On Ubuntu, you need to add the ppa repository before installing the tool, as shown below:

$ sudo apt-get install software-properties-common
$ sudo apt-add-repository ppa:ansible/ansible

$ sudo apt-get update
$ sudo apt-get install ansible

The Ansible documentation encourages Debian users to access the Ubuntu repository to obtain Ansible. You need to add the following line to /etc/apt/sources.list:

deb http://ppa.launchpad.net/ansible/ansible/ubuntu trusty main

You can then install the software using the following commands:

$ sudo apt-get update
$ sudo apt-get install ansible

The Parabola GNU/Linux-libre distribution is a derivative of Arch Linux, without the binary blobs. You can install Ansible using the pacman utility:

$ pacman -S ansible

The latest Ansible version 2.2 (as of date) is what we will use in this article. Ansible is also available for BSD variants, Mac OS X, and Windows. You are encouraged to refer to the Ansible documentation for more information.

Virtualisation

Ansible can be used to provision new machines and also configure them. Instead of using bare metal machines, you can create multiple Virtual Machines (VMs) on your system. Lots of Free and Open Source Software (F/OSS) virtualization software is available.

QEMU is a machine emulator and virtualiser. It can also use host CPU support to run guest VMs for better performance. It is written by Fabrice Bellard, and released under the GNU General Public License (GPL). You can install it on Parabola GNU/Linux-libre, using the following command:

$ sudo pacman -S qemu

KVM or Kernel-based Virtual Machine (KVM) has direct support in the Linux kernel. It requires hardware support to be able to run guest operating systems. It is written in C, and is released under the GNU General Public License.

You need to check if your hardware first supports KVM. The ‘lscpu’ command will show an entry for ‘Virtualization’ if there is hardware support. For example:

$ lscpu

Architecture:          x86_64
CPU op-mode(s):        32-bit, 64-bit
Byte Order:            Little Endian
CPU(s):                4
On-line CPU(s) list:   0-3
Thread(s) per core:    2
Core(s) per socket:    2
Socket(s):             1
NUMA node(s):          1
Vendor ID:             GenuineIntel
CPU family:            6
Model:                 78
Model name:            Intel(R) Core(TM) i5-6200U CPU @ 2.30GHz
Stepping:              3
CPU MHz:               2275.341
CPU max MHz:           2800.0000
CPU min MHz:           400.0000
BogoMIPS:              4801.00
Virtualization:        VT-x
L1d cache:             32K
L1i cache:             32K
L2 cache:              256K
L3 cache:              3072K
NUMA node0 CPU(s):     0-3
Flags:                 fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx pdpe1gb rdtscp lm constant_tsc art arch_perfmon pebs bts rep_good nopl xtopology nonstop_tsc aperfmperf eagerfpu pni pclmulqdq dtes64 monitor ds_cpl vmx est tm2 ssse3 sdbg fma cx16 xtpr pdcm pcid sse4_1 sse4_2 x2apic movbe popcnt tsc_deadline_timer aes xsave avx f16c rdrand lahf_lm abm 3dnowprefetch epb intel_pt tpr_shadow vnmi flexpriority ept vpid fsgsbase tsc_adjust bmi1 avx2 smep bmi2 erms invpcid mpx rdseed adx smap clflushopt xsaveopt xsavec xgetbv1 xsaves dtherm ida arat pln pts hwp hwp_notify hwp_act_window hwp_epp

You can also check the /proc/cpuinfo output as shown below:

$ grep -E "(vmx|svm)" --color=always /proc/cpuinfo

flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx pdpe1gb rdtscp lm constant_tsc art arch_perfmon pebs bts rep_good nopl xtopology nonstop_tsc aperfmperf eagerfpu pni pclmulqdq dtes64 monitor ds_cpl vmx est tm2 ssse3 sdbg fma cx16 xtpr pdcm pcid sse4_1 sse4_2 x2apic movbe popcnt tsc_deadline_timer aes xsave avx f16c rdrand lahf_lm abm 3dnowprefetch epb intel_pt tpr_shadow vnmi flexpriority ept vpid fsgsbase tsc_adjust bmi1 avx2 smep bmi2 erms invpcid mpx rdseed adx smap clflushopt xsaveopt xsavec xgetbv1 xsaves dtherm ida arat pln pts hwp hwp_notify hwp_act_window hwp_epp

The Libvirt project provides APIs to manage guest machines on KVM, QEMU and other virtualisation software. It is written in C, and is released under the GNU Lesser GPL. The Virtual Machine Manager (VMM) provides a graphical user interface for managing the guest VMs and is written in Python.

You can install all this software on Parabola GNU/Linux-libre using the following command:

$ sudo pacman -S libvirt virt-manager

A screenshot of Virtual Machine Manager is provided below:

Virtual Machine Manager

Check your distribution documentation to install the appropriate virtualisation software packages.

You can use the VMM to create a new virtual machine, and install a GNU/Linux distribution using an .iso image. You can specify RAM, disk size and follow the installation steps for your particular distro. You can also import an existing .qcow2 disk image to use it as a virtual machine.

Ansible with libvirt-VM

The version of Ansible used for this article is given below:

$ ansible --version

ansible 2.2.1.0
  config file = /etc/ansible/ansible.cfg
  configured module search path = Default w/o overrides

If you have the sshd daemon running on your local machine, you can use Ansible to test it. For example, a ping test on the localhost is shown below:

$ ansible localhost -m ping

localhost | SUCCESS => {
    "changed": false, 
    "ping": "pong"
}

You can also check how long the system has been up and running using the following command:

$ ansible localhost -a uptime

localhost | SUCCESS | rc=0 >>
 11:00:20 up  4:09,  0 users,  load average: 0.18, 0.14, 0.11

You can execute a shell command on the remote machine (localhost, in this case) as illustrated below:

$  ansible localhost -a "date"

localhost | SUCCESS | rc=0 >>
Sun Feb  5 11:24:53 IST 2017

The ‘setup’ command provides details of the remote target machine. A snippet output is provided below:

$  ansible localhost -m setup

localhost | SUCCESS => {
    "ansible_facts": {
        "ansible_all_ipv4_addresses": [
            "192.168.10.1", 
            "192.168.5.6"
        ], 
        "ansible_all_ipv6_addresses": [
            "fe90::fc24:ff:feb9:cb61", 
            "ff80::5846:fac1:6afc:2e30"
        ], 
        "ansible_architecture": "x86_64", 
        "ansible_bios_date": "06/12/2016", 
        "ansible_bios_version": "R00ET45W (1.20 )", 
        "ansible_cmdline": {
            "BOOT_IMAGE": "/vmlinuz-linux-libre", 
            "cryptdevice": "/dev/sda1:cryptroot", 
            "quiet": true, 
            "root": "/dev/mapper/cryptroot", 
            "rw": true
        }, 
        ....

An Ubuntu 15.04 instance with VMM is used in the following examples with Ansible. The IP address of the instance is added to /etc/hosts:

192.168.122.250 ubuntu

The /etc/ansible/hosts file contains the following:

ubuntu

You can now do a ping test from the host to the Ubuntu VM using the following command sequence for the user ‘xetex’:

$  ansible ubuntu -m ping -u xetex --ask-pass
SSH password: 
ubuntu | SUCCESS => {
    "changed": false, 
    "ping": "pong"
}

To avoid prompting for the password, you can add the localhost public SSH key to the VM, as follows:

$  ssh-copy-id -i ~/.ssh/id_rsa.pub xetex@ubuntu

/usr/bin/ssh-copy-id: INFO: Source of key(s) to be installed: "/home/user/.ssh/id_rsa.pub"
/usr/bin/ssh-copy-id: INFO: attempting to log in with the new key(s), to filter out any that are already installed
/usr/bin/ssh-copy-id: INFO: 1 key(s) remain to be installed -- if you are prompted now it is to install the new keys
xetex@ubuntu's password: 

Number of key(s) added: 1

Now try logging into the machine, with ssh xetex@ubuntu and check to make sure that only the key you wanted was added.

You can now issue the following command to get the same result:

$ ansible ubuntu -m ping -u xetex

ubuntu | SUCCESS => {
    "changed": false, 
    "ping": "pong"
}

For the Ubuntu system, you can also add the defined user in the /etc/ansible/hosts file as follows:

ubuntu ansible_ssh_host=ubuntu ansible_ssh_user=xetex

The ping command is now simplified to:

$  ansible ubuntu -m ping

ubuntu | SUCCESS => {
    "changed": false, 
    "ping": "pong"
}

You can now try the earlier Ansible commands on the target Ubuntu VM as illustrated below:

$ ansible ubuntu -a uptime

ubuntu | SUCCESS | rc=0 >>
 12:32:14 up 25 min,  3 users,  load average: 0.02, 0.07, 0.06

$ ansible ubuntu -a date

ubuntu | SUCCESS | rc=0 >>
Sun Feb  5 12:32:45 IST 2017

$  ansible ubuntu -m setup
ubuntu | SUCCESS => {
    "ansible_facts": {
        "ansible_all_ipv4_addresses": [
            "192.168.122.250"
        ], 
        "ansible_all_ipv6_addresses": [
            "ff20::5034:ff:fa9f:6123"
        ], 
        "ansible_architecture": "x86_64", 
        "ansible_bios_date": "04/01/2014", 
        "ansible_bios_version": "1.10.1-20151022_124906-anatol", 
        "ansible_cmdline": {
            "BOOT_IMAGE": "/boot/vmlinuz-3.19.0-15-generic", 
            "quiet": true, 
            "ro": true, 
            "root": "UUID=f43c2c72-5bc7-4a97-9a43-12e634ae232af", 
            "splash": true, 
            "vt.handoff": "7"
        }, 
        ...

I have installed and configured StumpWM (v1.0.0) on Parabola GNU/Linux-libre. StumpWM is a keyboard driven, tiling Window Manager written in Common Lisp (CL). It is thus very hackable, and you can customize its configuration by writing Lisp code and reload it dynamically. The installation and configuration steps are provided below for reference.

Firstly, the Steel Bank Common Lisp (SBCL) compiler needs to be installed on the system using Pacman.

$ sudo pacman -S sbcl 
$ sudo pacman -S acpi xtrlock emacs mate-terminal iceweasel

The acpi utility is installed to show the battery status in the modeline. The xtrlock utility is used to lock the screen. When the screen is locked, you will need to enter your user password to unlock the same. GNU Emacs, Iceweasel and Mate-terminal software packages are also installed. Quicklisp is a package manager for Common Lisp and is used to install the StumpWM dependencies with SBCL.

An installation directory is created at ~/downloads/quicklisp, and the quicklisplisp file is fetched as follows:

$ mkdir ~/downloads/quicklisp
$ cd ~/downloads/quicklisp
$ curl -O https://beta.quicklisp.org/quicklisp.lisp

The quicklisp.lisp file is loaded using the SBCL interpreter. The installation steps and output are shown below:

~/downloads/quicklisp $ sbcl --load quicklisp.lisp 

    This is SBCL 1.3.17, an implementation of ANSI Common Lisp.
    More information about SBCL is available at <http://www.sbcl.org/>.

    SBCL is free software, provided as is, with absolutely no warranty.
    It is mostly in the public domain; some portions are provided under
    BSD-style licenses.  See the CREDITS and COPYING files in the
    distribution for more information.

      ==== quicklisp quickstart 2015-01-28 loaded ====

        To continue with installation, evaluate: (quicklisp-quickstart:install)

        For installation options, evaluate: (quicklisp-quickstart:help)

* (quicklisp-quickstart:install)

    ; Fetching #<URL "http://beta.quicklisp.org/client/quicklisp.sexp">
    ; 0.82KB
    ==================================================
    838 bytes in 0.00 seconds (0.00KB/sec)
    ; Fetching #<URL "http://beta.quicklisp.org/client/2017-03-06/quicklisp.tar">
    ; 250.00KB
    ==================================================
    256,000 bytes in 0.16 seconds (1602.56KB/sec)
    ; Fetching #<URL "http://beta.quicklisp.org/client/2015-09-24/setup.lisp">
    ; 4.94KB
    ==================================================
    5,054 bytes in 0.00 seconds (0.00KB/sec)
    ; Fetching #<URL "http://beta.quicklisp.org/asdf/2.26/asdf.lisp">
    ; 194.07KB
    ==================================================
    198,729 bytes in 0.09 seconds (2086.79KB/sec)
    ; Fetching #<URL "http://beta.quicklisp.org/dist/quicklisp.txt">
    ; 0.40KB
    ==================================================
    408 bytes in 0.00 seconds (0.00KB/sec)
    Installing dist "quicklisp" version "2017-07-25".
    ; Fetching #<URL "http://beta.quicklisp.org/dist/quicklisp/2017-07-25/releases.txt">
    ; 372.80KB
    ==================================================
    381,744 bytes in 0.15 seconds (2485.31KB/sec)
    ; Fetching #<URL "http://beta.quicklisp.org/dist/quicklisp/2017-07-25/systems.txt">
    ; 241.23KB
    ==================================================
    247,022 bytes in 0.10 seconds (2297.45KB/sec)

      ==== quicklisp installed ====

        To load a system, use: (ql:quickload "system-name")

        To find systems, use: (ql:system-apropos "term")

        To load Quicklisp every time you start Lisp, use: (ql:add-to-init-file)

        For more information, see http://www.quicklisp.org/beta/

    NIL

* (ql:add-to-init-file)

    I will append the following lines to #P"/home/guest/.sbclrc":

      ;;; The following lines added by ql:add-to-init-file:
      #-quicklisp
      (let ((quicklisp-init (merge-pathnames "quicklisp/setup.lisp"
                                             (user-homedir-pathname))))
        (when (probe-file quicklisp-init)
          (load quicklisp-init)))

    Press Enter to continue.

    #P"/home/guest/.sbclrc"

* (ql:quickload "clx")

    To load "clx":
      Install 1 Quicklisp release:
        clx
    ; Fetching #<URL "http://beta.quicklisp.org/archive/clx/2017-06-30/clx-20170630-git.tgz">
    ; 452.92KB
    ==================================================
    463,786 bytes in 0.12 seconds (3806.02KB/sec)
    ; Loading "clx"
    [package xlib]....................................
    ..................................................
    ..................................................
    ..................................................
    ..................................................
    ..................................................
    ..................................................
    ..................................................
    [package xlib/glx]................................
    [package xlib/gl].................................
    [package xlib/dpms]...............................
    [package xlib/xtest]..............................
    [package xlib/xinerama]...........................
    [package xlib-demo/clclock].......................
    [package xlib-demo/clipboard].....................
    [package xlib-demo/demos].........................
    [package xlib-demo/gl-test].......................
    [package xlib-demo/mandel].............
    ("clx")

* (ql:quickload "cl-ppcre")

    To load "cl-ppcre":
      Install 1 Quicklisp release:
        cl-ppcre
    ; Fetching #<URL "http://beta.quicklisp.org/archive/cl-ppcre/2015-09-23/cl-ppcre-2.0.11.tgz">
    ; 156.08KB
    ==================================================
    159,829 bytes in 0.08 seconds (1903.45KB/sec)
    ; Loading "cl-ppcre"
    [package cl-ppcre]................................
    ................
    ("cl-ppcre")

* (ql:quickload "alexandria")

    To load "alexandria":
      Install 1 Quicklisp release:
        alexandria
    ; Fetching #<URL "http://beta.quicklisp.org/archive/alexandria/2017-06-30/alexandria-20170630-git.tgz">
    ; 49.97KB
    ==================================================
    51,168 bytes in 0.04 seconds (1135.65KB/sec)
    ; Loading "alexandria"
    [package alexandria.0.dev].......................
    ("alexandria")

* (exit)

~/downloads/quicklisp $ 

The StumpWM v1.0.0 sources are fetched to ~/downloads/stumpwm directory, extracted and compiled using the following steps:

$ mkdir ~/downloads/stumpwm
$ cd ~/downloads/stumpwm
$ wget https://github.com/stumpwm/stumpwm/archive/1.0.0.tar.gz
$ tar xzvf 1.0.0.tar.gz
$ cd stumpwm-1.0.0/
$ ./autogen.sh 
$ ./configure 
$ make

You can then install the built stumpwm binary, which will get copied to /usr/local/bin/stumpwm as shown below:

$ sudo make install

$ which stumpwm
/usr/local/bin/stumpwm

I use the Lightweight X11 Display Manager (lxdm), and thus created a /usr/share/xsessions/stumpwm.desktop file to login to StumpWM from the login manager:

[Desktop Entry]
Encoding=UTF-8
Name=StumpWM
Comment=StumpWM
Exec=/usr/local/bin/stumpwm
Type=Application

A sample ~/.stumpwmrc configuration file is given below (should be self-explanatory):

;; -*-lisp-*-

(in-package :stumpwm)

;; Startup message: display the machine's name
(setf *startup-message* (machine-instance))

;; Turn on the modeline
(if (not (head-mode-line (current-head)))
    (toggle-mode-line (current-screen) (current-head)))

;; Lock screen
(define-key *root-map* (kbd "L") "exec xtrlock")

;; I like messages to be centered on the screen.
(setf *message-window-gravity* :center)
(setf *input-window-gravity* :center)

;; I thought that this mode-line was fabulous!
(defvar *battery-status-command*
  "acpi -b | awk -F '[ ,]' '{printf \"%s%s\", $3, $5}' | sed s/Discharging/\-/ | sed s/Unknown// | sed s/Full// | sed s/Charging/+/")

(defvar *vol-status-command*
  "amixer get Master | grep \"[[:digit:]]\\+%\" -o | tr -d \"\\n\"")

(setf *screen-mode-line-format*
      (list "%w [^B%n^b] ^>"
      '(:eval (run-shell-command *battery-status-command* t))
      " | Vol. "
      '(:eval (run-shell-command *vol-status-command* t))
      " | %d"))

;; urxvt
(define-key *root-map* (kbd "c") 
  "exec urxvt +sb -fn \"xft:Ubuntu Mono:pixelsize=15,style=regular\"")

;; Window information format
(setf *window-info-format* "%wx%h %n (%t - %c)")

;; Window format
(setf *window-format* "%m%n%s%10t")

;; Emacs
(defvar *emacs-command* nil
  "Start an emacs client frame. Starts an emacs daemon if necessary.")
(setf *emacs-command* "bash -c -i 'emacsclient -c -a \"\"'")

(define-key *root-map* (kbd "e") "run-emacs")

(defcommand run-emacs () ()
    (run-shell-command (concat "exec " *emacs-command*)))

;; iceweasel
(defcommand iceweasel-browser () ()
  "run iceweasel"
  (run-or-raise "iceweasel" '(:instance "iceweasel")))
(define-key *root-map* (kbd "b") "iceweasel-browser")

;; mate-terminal
(defcommand mate-terminal () ()
  (run-or-raise "mate-terminal --hide-menubar" '(:class "mate-terminal")))

(define-key *root-map* (kbd "C") "mate-terminal")

;; Clear rules
(clear-window-placement-rules)

(define-frame-preference "Default"
  ;; frame raise lock (lock AND raise == jumpto)
  (0 t   t :instance "emacs")
  (1 t   t :instance "iceweasel-browser"))

;; dvorak and தமிழ்
(stumpwm:run-shell-command "sh -c 'setxkbmap us,in dvorak,tam_unicode grp:ctrls_toggle'")

;; Start default applications
(run-emacs)
(iceweasel-browser)

On logging into StumpWM, GNU Emacs and Iceweasel browser are opened automatically. I also switch between English and Tamil keyboard layouts when required, and the two Control keys are used to toggle between them. A StumpWM screenshot is shown below:

StumpWM screenshot

You are encouraged to read the StumpWM manual to know more about its usage and configuration.

I had organized a one-day workshop based on my book i want 2 do project. tell me wat 2 do at reserved-bit, Pune on Saturday, June 3, 2017. Thanks to Nisha Poyarekar and Siddhesh Poyarekar for providing their makerspace as venue for the workshop.

Workshop in progress

The objective of the workshop is to share the methodology and best practices on working with Free and Open Source Software projects, and also to guide the participants on career options. Although there is plenty of free documentation (including my own) available on the subject, some people prefer a formal, systematic approach to learning in a private coaching environment. Hence, this workshop is tailored for the same and is made a paid workshop, similar to personal tutoring.

The book has been in circulation for many years. So, I had to give two options - pay only for the workshop, or pay for both the workshop and the book (if they have not already bought the book). I have also kept additional copies of my book at the reserved-bit makerspace if people are interested. I had covered the following topics:

  1. Careers with F/OSS
  2. “i want 2 do project. tell me wat 2 do” best practices
  3. Introduction to Git
  4. Hands-on problem solving exercises
  5. Real-world project example
  6. Goal-driven development

The feedback has been positive and I am sharing the same below:

Feedback 1
Feedback 2
Feedback 3
Feedback 4
Feedback 5

If you are interested in attending such a workshop, do write to me: author at shakthimaan dot com.

Introduction

I attended Rootconf 2017 on May 11-12, 2017 at MLR Convention Centre, JP Nagar, Bengaluru.

Sponsors

The event had two parallel tracks, the Rootconf track at the main conference hall, and the DevConf.in track sponsored by Red Hat. A number of infrastructure hosting companies had sponsored the event. The selected talks had good diversity with topics ranging from monitoring, databases, networking, security etc.

Day I

The day’s proceedings began with the talk on the “State of open source monitoring landscape” by Bernd Erk. He gave a good overview of the available monitoring tools. He is involved in the Icinga2 project, which is re-written in C++ for performance. It is an alternative for the Nagios tool.

Aditya Patawari then provided an introduction to Kubernetes with a demo, as Spencer Krum could not make it to his talk on the “Anatomy of an alert”. After a short break, I attended “A little bot for big cause” session by Pooja Shah. The talk basically demonstrated how to create a bot by integrating various APIs.

I attended Manan Bharara’s talk on “Monitoring performance critical applications” where he had explained how they had used Clojure for monitoring at Otto in Germany. The presentation was very good, and the Oscillator Clojure code is available. He had provided good illustrations and code snippets in his presentation.

Manan Bharara' oscillator visualization

A number of stalls were available in the hallway, and I spent considerable time talking with the sponsors. At the Red Hat stall, I learnt some of the features and use cases for the Foreman provisioning tool. I was pictured with the Foreman helmet:

Shakthimaan with Foreman helmet

After lunch, I attended “SELinux: A deep dive” by Rejy M Cyriac. He gave an excellent introduction to SELinux, explaining the underlying concepts and rationale for the same. We also did basic hands-on exercises, but, I could not attend the last section as I had to move to the main conference hall for the flash talks. I sang the Nursery Rhymes version for DevOps. The video is available.

We then headed to the Rootconf party sponsored by Go-Jek!

Day II

The second day started with the talk by Kunal Grover on “Failure resilient architecture with microservice dependencies”. He presented the ways by which disruptions are performed in cloud infrastructure and how recovery mechanisms are tested. Ramya A then gave an interesting talk on “Asynchronous integration tests for microservices”. She had explained the pact.io family of frameworks to support Consumer Driven Contracts testing. Laxmi Nagarajan then gave a high level overview of “Capacity planning for AWS and configuring AWS autoscaling policies” sharing her Industry experience. After a short break, Colin Charles presented his story on “Capacity planning for your data stores” citing real world examples.

I then moved to the DevConf.in track to attend Ruchi Singh’s talk on “Migration of 300 microservices from AWS cloud to Snapdeal cloud”. The time was too short for such a talk, and it was an overview. I would have liked to see more details, given that they use Chef for provisioning and handle more than 16 TB of data as mentioned by Ruchi Singh.

After a quick lunch, I attended “Adventures in Postgres management” by Ramanan Balakrishnan. It was a good technical talk going over the failures and experiences learnt. After the talk I went to the hallway to listen to the Off-The-Record (OTR) session on mistakes to avoid when planning your infrastructure.

OTR session

I returned to the main conference hall for doing some stretching exercises. In my opinion, all conferences should make this session mandatory, especially after you have been sitting for a long period of time. This session was then followed by the “Working with Secrets” talk by Shrey Agarwal, who gave an overview of using Hashicorp’s Vault for managing passwords. It was a very short introduction on the topic.

After a short beverage break, Toshaan Bharvani presented “Living with SELinux”, which was an excellent presentation on the subject. The initial slides had some material that Rejy M Cyriac had introduced the previous day, but, the content and presentation were good. With proper SELinux policies, he said that root in his production servers cannot do much. Rejy and Toshaan both asked people to use permissive mode instead of disabling SELinux altogether, so that you at least know what is being audited.

The last talk of the day was by Anurag Bhatia on “Understanding eyeball routing via RIPE Atlas”. He gave an overview of the RIPE Atlas project and how network metrics are measured and monitored wherever the RIPE device is installed. It is a TP-Link hardware whose firmware can be flashed using Libre software. Unfortunately, at this point the source of the firmware is not released as Free and Open Source Software. I was told that there is still an ongoing discussion on the same.

Conclusion

The talk by Manan Bharara and Rejy’s SELinux workshop were the highlights for day one for me. The content on the second day was much better and had greater depth in my opinion. Overall, it was a useful technical conference, and a good place to meet like-minded people.

I would like to thank Aerospike for sponsoring me to attend the conference.

[Published in Open Source For You (OSFY) magazine, June 2016 edition.]

We have covered a lot of ground in our quest to learn the intricacies of GNU Emacs and we trust that you have found your long journey fulfilling. This is the last article in the series.

Introduction

The GNU Emacs editor is written using Emacs Lisp and the C programming language. You can use GNU Emacs as an editor without knowing Emacs Lisp, but, being familiar with it will help you customize it or extend it to your needs. Emacs Lisp is a dialect of the Lisp programming language and is inspired by Maclisp and Common Lisp. The source files end with the filename extension .el and the byte-compiled files end with the .elc filename extension. You can also write scripts using Emacs Lisp and execute them as a batch operation. The code can thus be executed from the command line or from an executable file. Byte-compiling the source files can help you speed up the execution.

Comments

All comments begin with a semi-colon. An Emacs Lisp file usually has the following sections in the code, specified with three semi-colons. These always start in the left margin as shown below:

;;; Module --- Summary

;;; Commentary:

;;; Code:

;;; Module ends here

All comments outside functions begin with two semi-colons. The contents of the scratch buffer, shown below, are an example:

;; This buffer is for notes you don't want to save, and for Lisp evaluation.
;; If you want to create a file, visit that file with C-x C-f,
;; then enter the text in that file's own buffer.

Comments inside functions use a single semi-colon, and if they span multiple lines, they must be neatly aligned. For example:

...
(let* ((sexp (read (current-buffer)))    ; using `read' here
                                         ; easier than regexp
                                         ; matching, esp. with
                                         ; different forms of
                                         ; MONTH
 ...
 ))

Literals

The basic data types are available in Emacs Lisp. Numbers can be represented by integers or floats. Integers can have their sign representation before the digit (+1, -2). Floating point numbers can be represented using a decimal point (3.1415) or with an exponent (314.15e-2). A character (S) is represented by its integer code (83), while a string is a list of characters enclosed within double quotes (“A string”).

A symbol is an object with a name. A keyword symbol is one that starts with a colon (:). A vector is an array and can contain different types ([1 “two” :three]). The Boolean values for true and false are ’t’ and ‘nil’ respectively. A cons cell is an object with two slots. The first slot is called the CAR (Contents of the Address part of the Register number) and the second slot is called the CDR (Contents of the Decrement part of the Register number). A list is a series of linked cons cells. For example, in the list ‘(A B)’, the CAR is A and the CDR is B.

Sexp

Emacs Lisp uses prefix notation which consists of an operation followed by operands (arguments). All programs are represented as symbolic expressions (sexp). For example, the ‘+’ operation is applied to its arguments ‘1’ and ‘2’ in the following sexp:

(+ 1 2)

If you copy the above code in an Emacs buffer, you can evaluate the same by placing the cursor at the end of the expression and using the C-x C-e shortcut. The output ‘3’ will be displayed in the minibuffer.

You can also have nested sexps, wherein the innermost expressions are evaluated first.

(+ 1 (* 2 3))

Similar to the ‘+’ operation, you can use pre-defined keywords or functions in Emacs Lisp programs. The format function is similar to your printf statement in the C programming language.

(format "Hello, World!")

You can store a value in a variable using the setq operation as shown below:

(setq IST "Indian Standard Time")

IST     ; No enclosing parenthesis!
        ; produces "Indian Standard Time"

You can find the data type, using the type-of function. A few examples are shown below:

(type-of 1)                ; produces integer

(type-of 3.1415)           ; produces float

(type-of "A string")       ; produces string

(type-of :foo)             ; produces symbol

(type-of t)                ; produces symbol

(type-of '(1 2))           ; produces cons

(type-of [1 "two" :three]) ; produces vector

In the Bash shell, you can escape evaluation by using the backslash. Similarly, you can prevent the Emacs Lisp interpreter from evaluating an expression using quotes. For example:

(+ 1 2)     ; produces 3

'(+ 1 2)    ; produces the list (+ 1 2)

progn

The progn statement is used to execute several sexps. For example:

(progn
  (setq title "Introduction to Emacs Lisp")
  (setq author "Robert J. Chassell")
  (format "%s by %s" title author))  ; produces "Introduction to Emacs Lisp by Robert J. Chassell"

Functions

You can define your own function using the defun built-in keyword. The ‘say’ function simply prints the string “Hello, World!”" in the following example:

(defun say ()
  (format "Hello, World!"))

In order to execute the function, invoke it inside parenthesis as shown below:

(say)    ; produces "Hello, World!"

Arguments can be passed to a function. The ‘square’ function is demonstrated below:

(defun square (x)
  (* x x))

(square 3)  ; produces 9

You can also store a list of names and retrieve them using the car and cdr functions as illustrated below:

(setq teams '("GL" "DD" "KKR" "MI" "SRH" "RPS" "RCB" "KXIP"))

(car teams)       ; produces "GL"

(cdr teams)       ; produces "DD" "KKR" "MI" "SRH" "RPS" "RCB" "KXIP"

(car (cdr teams)) ; produces "DD"

let

The let statement allows you to bind a value to a variable in a local context. GNU Emacs 24 has lexical binding support. An example is given below:

(defun hello (name)
  (let ((new-name (concat "Hello " name)))
    new-name
      ))

(hello "Mark")   ; produces "Hello Mark"

Conditions

Conditions can be checked using the if and cond statements. The if statement takes a condition, and if the condition evaluates to true, the sexp immediately following the condition is executed. Otherwise, the else-form is evaluated. The syntax is

(if condition then-form else-form)

An example:

(if t
    (format "True")
  (format "False"))  ; produces "True"

The when condition statement is a variant of the if statement written as a macro. Macros are an important feature of Lisp programming that allow you to extend the language. They are primarily code generators. If the ‘when’ condition is true, then it evaluates the then-forms. The syntax and an example are given below:

(when condition then-forms)

(when t
  (print "Abracadabra!"))  ; produces "Abracadabra!"

Looping

You can use the while statement to perform looping in Emacs Lisp. The body of the ‘while’ loop is executed as long as the condition is true. An example that prints the numbers 0 to 4 is shown below:

(progn
  (setq i 0)
  (while (< i 5)
    (print i)
    (setq i (+ i 1))))

The dolist and dotimes macros can also be used for looping. The other approach is to use recursion. A literal definition in Emacs Lisp for computing the Fibonacci series is given below:

(defun fib (n)
  (cond ((= n 0) 0)
        ((= n 1) 1)
        (t (+ (fib (- n 1))
              (fib (- n 2))))))

The cond control structure takes a series of clauses, each has a condition and a body-form. If any condition evaluates to true, the body-form is executed. Otherwise, it proceeds to the next clause.

An iterative version of the Fibonacci series is given below:

(defun fib (n)
  (fib-helper n 0 1))

(defun fib-helper (n a b)
  (if (= n 0)
      a
    (fib-helper (- n 1) b (+ a b))))

(fib 10) ; produces 55

The Emacs Lisp Cookbook is very handy https://www.emacswiki.org/emacs/ElispCookbook.

You are encouraged to read ‘An Introduction to Programming in Emacs Lisp’ at https://www.gnu.org/software/emacs/manual/eintr.html and the Emacs Lisp Reference Manual at https://www.gnu.org/software/emacs/manual/elisp.html.

I had a chance to attend the Pune GNU/Linux Users Group (PLUG) ClipArt Hackathon on Sunday, March 12, 2017 at Bhaskaracharya Pratishthana. The hackathon was an initiative to start organizing different kinds of F/OSS events.

Gaurav (“dexter”) Pant started the day’s proceedings with a quick demo of Inkscape. He also provided the participants with an Inkscape quick reference sheet. The advantage of creating Scalable Vector Graphics (SVG) is that it can be used on a web page as well as enlarged for printing on posters and banners, without any distortion.

I took this time to create some illustrations. The first image was a logo with the letter “S”, whose shape was modified to fit into a boundary. The stick-figure-push SVG is already available in Openclipart. I modified it for a stick-figure-pull image. Few icons were then created for use in web pages. Finally, I completed an SVG of “The Common Man”.

S logo

Stick figure pull

Envelope icon

Monitor icon

Telephone icon

The Common Man

All the images are available in Creative Commons Attribution-Share Alike 4.0 International license at Wikimedia ( https://commons.wikimedia.org/w/index.php?title=Special:ListFiles/Shakthimaan ).

The hackathon was a good initiative to encourage creative contributions to F/OSS. I was happy to see a number of excellent designers who were creating beautiful SVGs. Participants used both GIMP and Inkscape for their work. It was also good to have met the PLUG members.

Group photo
PC: Siddharth Subramaniam

Such events help grow the activities in a region, and also showcases the diversity in a F/OSS group.

[Published in Open Source For You (OSFY) magazine, May 2016 edition.]

Keeping notes, maintaining to-do lists, planning projects and author documents with a fast and effective plain-text system - that’s what Emacs Org mode helps you do. There’s even a mobile version available as FOSS so that your favourite note-taking tool remains with you always.

In this next article in the GNU Emacs series, let’s learn about Org mode.

Introduction

Org-mode is an outline, major mode in GNU Emacs that can be used for taking notes, planning and documentation. It was written by Carsten Dominik in 2003. The Org file is written in plain text and markdown, and it can be exported into multiple output formats (TeX, HTML, PDF, etc.). Org-mode ships with GNU Emacs and this tutorial uses GNU Emacs 24.4.1 (M-x version) and Org mode version 8.2.10 (M-x org-version).

Structure

An Org file ends with the .org filename extension. The top-level headlines are marked using an asterisk (‘*’). For example, a college student may have the following high-level topics:

* College
* Home
* Recreation

You can create topics in the sub-section with more asterisks. For example:

* College
** Assignments
** Lab
** Exams
* Home
* Recreation

If you wish to hide the multiple asterisks at the sub-section level, you can add the following to your ~/.emacs:

(setq org-hide-leading-stars t)

The resultant Org file will look like what follows:

* College
 * Assignments
 * Lab
 * Exams
* Home
* Recreation

You can add unnumbered (use minus or plus symbols) or numbered lists as shown below:

* College
 * Assignments
 * Lab
   1. Compiler Design
   2. Programming Languages
 * Exams
   - Mathematics
   - Operating Systems
   - Databases
   - Compilers
* Home
* Recreation

You can cycle through the various levels in the Org file using the Tab and Shift-Tab keys.

A checkbox with a ‘/’ or a ‘%’ symbol can be used next to a topic name to indicate the completion status of the task. The lists under a topic can each have a checkbox too. Using C-c C-c will mark a checkbox for completion (‘X’) and will also update the statistics in the top-level checkbox. An example is shown below:

* College...
* Home [2/2]
  - [X] Read book
  - [X] Take print-out
* Recreation [50%]
  - [X] Read newspaper
  - [ ] Meditate

If a task is completed, you can move it out of the Org file using C-c C-x C-a. This will create an archive file with the completed items.

Planner

An Org mode file can be used as a planner. Each task can be marked with any of the following states - {TODO, DONE} using C-c C-t key combination. For example:

* TODO College...
* DONE Home [2/2]...
* Recreation [50%]...

You can also customize the TODO states depending on your workflow by setting org-todo-keywords in your Emacs startup file. For example:

(setq org-todo-keywords
  '((sequence "TODO(t)" "NEXT(n)" "STARTED(s)" "WAITING(w)" "|" "DONE(d)" "CANCELED(c)")))

The tasks can be scheduled using C-c C-s. A date is prompted for using the Calendar and is placed below the list entry. For example:

* TODO College
 * Assignments
 * Lab
   SCHEDULED: <2016-04-06 Wed>
   1. Compiler Design
   2. Programming Languages
 * Exams...

You can also add the time interval during which you intend to complete the task. The above example with a scheduled time is shown below:

* TODO College
 * Assignments
 * Lab
   SCHEDULED: <2016-04-06 Wed 14:00-16:00>
   1. Compiler Design
   2. Programming Languages
 * Exams...

A deadline can be added to a task using C-c C-d shortcut. An example is given below:

* TODO College
 * Assignments
 * Lab...
 * Exams
   DEADLINE: <2016-04-08 Fri>
   - Mathematics
   - Operating Systems
   - Databases
   - Compilers
* DONE Home [2/2]...
* Recreation [50%]...

You can have multiple Org files stored in your system, and you can instruct GNU Emacs where to find them by setting org-agenda-files in your Emacs start-up file as shown below:

(setq org-agenda-files (quote ("/tmp")))

Additionally, if you want an agenda view to see the scheduled items and deadlines, add the following to your GNU Emacs startup init file:

(define-key global-map "\C-ca" 'org-agenda)

In the Org file, when you press C-c a, the following agenda will show up in a new buffer:

Week-agenda (W14):
Monday      4 April 2016 W14
  test:       In   4 d.:  Exams
Tuesday     5 April 2016
Wednesday   6 April 2016
  test:       14:00-16:00 Scheduled:  Lab
Thursday    7 April 2016
Friday      8 April 2016
  test:       Deadline:   Exams
Saturday    9 April 2016
Sunday     10 April 2016

Tables

Org-mode has a built-in table editor which neatly aligns the column data. For example:

* TODO College...
* DONE Home [2/2]...
* Recreation [50%]
  - [X] Read newspaper
  - [ ] Meditate
  | Day       | Time | Status |
  |-----------+------+--------|
  | Monday    | 1.25 | Done   |
  | Tuesday   | 1.50 | Done   |
  | Wednesday |      |        |
  | Thursday  |      |        |
  | Friday    |      |        |
  | Saturday  |      |        |
  | Sunday    |      |        |

You can also use spreadsheet formula on these tables to perform calculations. For example:

* TODO College...
* DONE Home [2/2]...
* Recreation [50%]
  - [X] Read newspaper
  - [ ] Meditate
  | Day       | Time | Status |
  |-----------+------+--------|
  | Monday    | 1.25 | Done   |
  | Tuesday   | 1.50 | Done   |
  | Wednesday |      |        |
  | Thursday  |      |        |
  | Friday    |      |        |
  | Saturday  |      |        |
  | Sunday    |      |        |
  |-----------+------+--------|
  | Total     | 2.75 |        |
  #+TBLFM: @9$2=vsum(@2$2..@8$2)

Exporting

The Org file can be exported to multiple output formats (TeX, HTML, ASCII, PDF, etc.). Using C-c C-e will produce a buffer with the ‘Org Export Dispatcher’ menu to select an exporter. This is shown in the following figure:

Org Export Dispatcher

You can also write your own backend customisations to suit your needs.

Literate programming

Donald Knuth coined the term ‘Literate Programming’ in 1984. To quote him:

"I believe that the time is ripe for significantly better documentation of programs, and that we can best achieve this by considering programs to be works of literature. Hence, my title: ‘Literate Programming’.

Let us change our traditional attitude to the construction of programs: Instead of imagining that our main task is to instruct a computer what to do, let us concentrate rather on explaining to human beings what we want a computer to do."

Org mode supports this style of programming using Babel. You need to activate the support for the programming languages in your GNU Emacs startup file. For example, the following code snippet helps to execute Bash shell scripts.

(org-babel-do-load-languages
 'org-babel-load-languages
 '((sh . t)
   ))

Consider a shell command to find the disk usage. You can create an Org file, and enclose the command in a Babel code block as shown below:

#+BEGIN_SRC sh
  df -h
#+END_SRC

When you press C-c C-c on this code block, you will be prompted with the string “Evaluate this sh code block on your system? (yes/no).” If you input “yes”, the output is produced in a Results section as shown below:

#+RESULTS:
| Filesystem | Size | Used | Avail | Use% | Mounted        | on |
| udev       | 1.9G | 0    | 1.9G  |   0% | /dev           |    |
| tmpfs      | 384M | 6.0M | 378M  |   2% | /run           |    |
| /dev/sda1  | 913G | 75G  | 792G  |   9% | /              |    |
| tmpfs      | 1.9G | 57M  | 1.9G  |   3% | /dev/shm       |    |
| tmpfs      | 5.0M | 4.0K | 5.0M  |   1% | /run/lock      |    |
| tmpfs      | 1.9G | 0    | 1.9G  |   0% | /sys/fs/cgroup |    |
| tmpfs      | 384M | 64K  | 384M  |   1% | /run/user/1000 |    |

You can learn more on Babel from their web page http://orgmode.org/worg/org-contrib/babel/.

A mobile version of Org mode is also available as Free and Open Source Software. You can use a third party service to sync your Org files between your mobile and system. Since the files are plain text, they can also be revision controlled using Git or any version control software.

Please refer to the Org reference manual at http://orgmode.org/#docs for more tips, customisation options and documentation.

I attended PyCon Pune 2017 conference between February 16-17 (Thursday-Friday), 2017 at Amanora - The Ferns Hotels and Club, Pune.

Ferns Hotel

Day I

I reached the venue early in the morning, to be of any help to the volunteers. The projector had to be checked, and I used my laptop to test the same. After changing couple of cables and adapters, the clarity on the screen was good.

This event had a single track where everyone sat in one big hall. I welcome this change!

Honza Král started the conference with his keynote, titled “(My) OSS Life”. He shared his experiences and learning in the Free and Open Source Software (F/OSS) world. At present, he maintains the Python drivers for Elasticsearch.

Honza Král

The keynote was followed by Anand Chitipotu’s talk on “Writing Beautiful Code”. He illustrated code examples on how to write simple, elegant, readable Python code. The use of meaningful variable names, comments were emphasized a lot. It was a short list of collated points on basic mistakes that newbie programmers make, and how to effectively write beautiful code.

Florian Fuchs then spoke on “Hacking Mailing Lists - The Mailman 3 API Ecosystem”. He explained the new architecture and API with code examples. He has been hacking on the new Mailman 3 web UI and the Python API bindings.

After attending these talks, I made a visit to the three booths at the conference - Red Hat, Python Software Foundation and reserved-bit. I also signed copies of my book that people had brought.

After lunch, I attended the “i18n-ise Django Apps” talk by Sundeep Anand, where he showed the internationalization processes for a Django application. All the relevant file modifications and commands involved were demonstrated.

John ‘warthog9’ Hawley is a Perl guy and gave an interesting keynote on building your own hardware, and why you should do that. He explained the various challenges he had faced, the process involved in the same. He had exclusively designed and produced a “Battle Bunny” embedded micro-Python kit for the conference and development sprints.

The “Building Trust in Releases” talk by Nigel Babu was very informative. He explained four important aspects in release management - cadence, documentation, testing, and empathy. This was also an experience report on DevOps practices, and was quite detailed and useful.

The last keynote of the day was by a Physics teacher, Praveen Patil. He shared his exploration on using Python to teach Physics to high school students. He is actively involved in ExpEYES Project, teacher training programs and also contributes content to National Repository of Open Educational Resources (NROER).

Praveen Patil system setup

Day II

The morning keynote was by Katie Cunningham. She was initially testing the microphone and laptop by singing nursery rhymes. While the organizers decided to wait for people to arrive and settle down, there was time for lightning talks. So, I volunteered to start the day with the resounding “Nursery Rhymes”. After a couple of other lightning talks, Katie started her keynote on accessibility guidelines. It was quite an informative session.

“You can help develop Python - and you should!” talk by Stephen Turnbull was on the history of Python, PEP guidelines and the functioning of the community. I also had a chance to talk with him personally on the story of XEmacs.

Farhaan Bukhsh and Vivek Anand presented their Google Summer of Code (GSoC) work on the project Pagure, which is an alternative to GitHub and GitLab. They shared the past, present and future roadmap for the project. In the “Testing native binaries using CFFI” talk, Noufal Ibrahim demonstrated how to write Python bindings using CFFI.

After lunch, there was time for lightning talks. Different Python user group communities (PyDelhi, HydPy, PythonPune, PyLadies Pune) pitched about their work . I had prepared the sequel to “The Yet Another Lightning Talk” and requested the audience to sing on my behalf. The feedback was positive, as usual. The latest addition to the nursery rhyme is as follows:

Twinkle, Twinkle, unit tests,
How I wonder, where you exist!
I will write unit tests,
Until the project is laid to rest.

The afternoon keynote was by Nick Coghlan. He also shared his know-how on Free and Open Source Software and community best practices. “Django on Steroids - Lessons from Scale” by Sanket Saurav was a good technical, intermediate-level talk on Django customization, nginx settings, scaling and deployment.

The last keynote of the day and the conference was by Terri Oda on “Is OSS more secure?”. She presented the various dimensions on which one needs to answer the question. She concluded by saying that there is definitely more scope in F/OSS to be more secure given the number of people involved, and the transparency in the process.

Conclusion

The number of participants at the conference was more than five hundred, and they all came on a week day to attend! For a first-time event, that is quite impressive. This clearly shows that there is a demand for such events across India.

PyCon Pune 2017 group photo
PC: Kushal Das

Initially, there was a lot of resistance (follow the thread) to this event, including the name of the event. Communities are meant for innovation, and stifling is futile. You can learn a lot in a community, and there are guidelines and best practices that are followed.

It was a four day event with development sprints, and hence it had to be a PyCon. Legally, the Python Software Foundation allows using the name “PyCon” for regional events too. Given the context and the necessity, I am happy that the Python Software Foundation (PSF) got the message right, and understood the need for the conference and supported it in a big way!

The development sprints had a limited seating capacity, and the registration got over early. I was informed that there were hundred more requests for the development sprints, which again emphasizes the need for such events! I also had a chance to meet some of the #dgplug (irc.freenode.net) folks with whom I have been interacting online on IRC.

It does take a lot of effort to organize a conference, and I congratulate the PyCon Pune team on their first event.

[Published in Open Source For You (OSFY) magazine, April 2016 edition.]

This article in the GNU Emacs series takes readers on how to use HTML mode, do indentation, and use the Magit interface.

HTML mode

You can use HTML mode to effectively edit HTML and CSS files using GNU Emacs. To start the mode, use M-x html-mode. You will see the string ‘HTML’ in the mode line.

Default template

A default HTML template can be started by opening a test.html file, and using C-c C-t html. It will produce the following content:

<html>
  <head>
<title>

You will then be prompted with the string ‘Title:’ to input the title of the HTML page. After you type ‘Hello World’, the default template is written to the buffer, as follows:

<html>
  <head>
<title>Hello World</title>
</head>
<body>
<h1>Hello World</h1>

<address>
<a href="mailto:user@hostname">shakthi</a>
</address>
</body>
</html>

Tags

You can enter HTML tags using C-c C-t. GNU Emacs will prompt you with the available list of tags. A screenshot of the available tags is shown in Figure 1:

HTML tags

The anchor tag can be inserted using ‘a’. You will then receive a message prompt: ‘Attribute:’. You can provide the value as ‘href’. It will then prompt you for a value, and you can enter a URL, say, ‘http://www.shakthimaan.com’. The anchor tag will be constructed in the buffer as you input values in the mini-buffer. You will be prompted for more attributes. If you want to finish, simply hit the Enter key, and the anchor tag will be completed. The final output is shown below:

<a href="http://www.shakthimaan.com"></a>

You can insert a h2 tag by specifying the same after C-c C-t. You can also add any attributes, as required. Otherwise, simply hitting the Enter key will complete the tag. The rendered text is as follows:

<h2></h2>

You can insert images using the alt tag. You can specify the src attribute and a value for the same. It is also a good practice to specify the alt attribute for the image tag. An example is shown below:

<img alt="image" src="http://shakthimaan.com/images/ShakthiK-workshop-A4-poster.png">

Unordered lists can be created using C-c C-t followed by ‘ul’. It will then prompt you for any attributes that you want included in the tag. You can hit the Enter key, which will prompt you with the string ‘List item:’ to key in list values. An example of the output is shown below:

<ul>
  <li>One
    <li>Two
      <li>Three
</ul>

You can neatly align the code by highlighting the above text and indenting the region using C-M-\. The resultant output is shown below:

<ul>
  <li>One
  <li>Two
  <li>Three
</ul>

If you wish to comment out text, you can select the region and type M-q. The text is enclosed using “<!--” and “-->”. For example, the commented address tags in the above example look like what follows:

<!-- <address> -->
<!-- <a href="mailto:shakthi@achilles">shakthi</a> -->
<!-- </address> -->

A number of major modes exist for different programming environments. You are encouraged to try them out and customize them to your needs.

Accents

In HTML mode, you can insert special characters, accents, symbols and punctuation marks. These characters are mapped to Emacs shortcuts. Some of them are listed in the following table:

Shortcut Character
C-x 8 ’ a á
C-x 8 " e ë
C-x 8 / E Æ
C-x 8 3/4 ¾
C-x 8 C ©
C-x 8 L £
C-x 8 P
C-x 8 u µ
C-x 8 R ®
C-x / / ÷

Indentation

Consider the following paragraph:

“When we speak of free software, we are referring to freedom, not price. Our General Public Licenses are designed to make sure that you have the freedom to distribute copies of free software (and charge for this service if you wish), that you receive source code or can get it if you want it, that you can change the software or use pieces of it in new free programs; and that you know you can do these things.”

You can neatly fit the above text into 80 columns and 25 rows inside GNU Emacs using M-q. The result is shown below:

When we speak of free software, we are referring to freedom, not
price.  Our General Public Licenses are designed to make sure that you
have the freedom to distribute copies of free software (and charge for
this service if you wish), that you receive source code or can get it
if you want it, that you can change the software or use pieces of it
in new free programs; and that you know you can do these things.

You can also neatly indent regions using the C-M-\ shortcut. For example, look at the following HTML snippet:

<table>
<tr>
<td>Tamil Nadu</td>
<td>Chennai</td> 
</tr>
<tr>
<td>Karnataka</td>
<td>Bengaluru</td> 
</tr>
<tr>
<td>Punjab</td>
<td>Chandigarh</td> 
</tr>
</table>

After indenting the region with C-M-\, the resultant output is shown below:

<table>
  <tr>
    <td>Tamil Nadu</td>
    <td>Chennai</td> 
  </tr>
  <tr>
    <td>Karnataka</td>
    <td>Bengaluru</td> 
  </tr>
  <tr>
    <td>Punjab</td>
    <td>Chandigarh</td> 
  </tr>
</table>

If you have a long line which you would like to split, you can use the C-M-o shortcut. Consider the quote:

“When you’re running a startup, your competitors decide how hard you work.” ~ Paul Graham

If you keep the cursor after the comma, and use C-M-o, the result is shown below:

"When you're running a startup, 
                                your competitors decide how hard you work." ~ Paul Graham

Magit

Magit is a fantastic interface to Git inside GNU Emacs. There are many ways in which you can install Magit. To install from the Melpa repository, add the following to your ~/.emacs:

(require 'package)
(add-to-list 'package-archives
             '("melpa" . "http://melpa.org/packages/") t)

When you do M-x list-packages, you will see ‘magit’ in the list. You can press ‘i’ to mark Magit for installation, followed by ‘x’ to actually install it. This will install Magit in ~/.emacs.d/elpa. The version installed on my system is magit-20160303.502.

When you open any file inside GNU Emacs that is version controlled using Git, you can start the Magit interface using M-x magit-status. I have bound this key to C-x g shortcut in ~/.emacs using the following:

(global-set-key (kbd "C-x g") 'magit-status)

The default magit screenshot for the GNU Emacs project README file is shown in Figure 2.

Magit

Pressing ‘l’ followed by ‘l’ will produce the history log in the magit buffer. A screenshot is provided in Figure 3.

History

You can make changes to the project sources and stage them to the index using the ’s’ shortcut. You can unstage the changes using the ‘u’ shortcut. After making changes to a file, you need to use M-x magit-status to update the Magit buffer status.

A sample screenshot of the modified files and staged changes is shown in Figure 4.

Staged

You can hit TAB and Shift-TAB to cycle through the different sections in the Magit buffer. To commit a message, press ‘c’ followed by ‘c’. It will pop up a buffer where you can enter the commit message.

You can create and checkout branches using the ‘b’ shortcut. A screenshot of the magit branch pop-up menu is shown in Figure 5.

Branch

All the basic Git commands are supported in Magit - diffing, tagging, resetting, stashing, push-pull, merging and rebasing. You are encourged to read the Magit manual ( https://magit.vc/ ) to learn more.

« OLDER POSTS