news

2005 2006 2007 2008 2009 2010 2011 2012 2013 2014 2015 2016 FOSS documentation emacs fedora foss freedom gnome haskell install laptop photo ruby travel verilog vhdl vlsi xmonad


[Published in Open Source For You (OSFY) magazine, October 2015 edition.]

Internet Relay Chat (IRC) provides instant communication over the Web. It is generally used for one-to-one communication or group discussion forums, also known as channels. In this next article in the GNU Emacs series, we shall learn how to use GNU Emacs as an Internet Relay Chat (IRC) client.

What is IRC?

Internet Relay Chat (IRC) is a messaging protocol that was created by Jarkko Oikarinen in 1988. An IRC server has a number of channels (or rooms, as how they are called in other chat software) where both technical and non-technical discussions take place. Every user requires a nickname to chat. You will need to register your nickname with a password to identify yourself every time you log in to the IRC server. IRC can also be used for file sharing among users.

You can connect to any one of a number of IRC servers that are available worldwide, free. The most popular is Freenode (irc.freenode.net). The other IRC servers, to name a few, are IRCNet, OFTC, and EFNet. The Debian and Ubuntu projects have their own chat servers – irc.debian.org and irc.ubuntu.com. All IRC channel names begin with ‘#’. Some channels begin with ‘##’. Examples of channels are ##linux-india, #dgplug, #emacs, #ruby, #guile, #lisp, and #scheme. You can also create your own channels, or host your own IRC server. Some examples of free and open source IRC server software are IRCd, UnrealIRCd, and Synchronet. Each IRC channel has a topic that can include useful links relevant to the channel. Sometimes announcements, or news updates are also mentioned in the topic.

Basic commands

A number of commands can be given to the IRC server. A few of them are discussed below:

  1. /list is used to provide a list of all the available channels in a server. For example: /list in irc.freenode.net returned the following:

    ...
    #linod           1       
    ##welding        3       Welcome to ##Welding, We're a little bare at the moment, but will help if we can. Tutorials: https://www.youtube.com/channel/UCJAFY2kKKb5sg79yld7T3hA
    #drupal-ph       1       "Drupalista! Welcome to Philippine Drupal Users Group. Have a good time chatting. If you have a question, please don't ask to ask but fire up your question in very specific and constructive way! Please join #drupal or #drupal-support if no one is around"
    #orx-project     4       Orx: Portable Game Engine
    #tinkerforge     5       
    #osi             10      The Open Source Initiative
    #xampp           1       
    #guitar          8       
    #bitcoin-ar      3       Comunidad Bitcoin Argentina
    #LargeHadrosaurCollider 19      Welcome to the LHC, est. 2001 | http://www.largehadrosaurcollider.net | August Birthdays: Digby 08/21, Josh 08/31 | At night it is pitch black, often for months. | http://tinyurl.com/psgdagl
    * End of /LIST
  2. /msg NickServ REGISTER password e-mail is used to register your nickname to the IRC server. /msg NickServ IDENTIFY password is used to identify yourself to the server.

  3. /me message displays the message for a user. For example:

    /me says "Hello, World!"
    
     * mbuf says "Hello, World!"
  4. /whois nickname provides useful information for a user. For example:

     /whois mbuf
    
    * [mbuf] (~shakthi@123.123.123.123): Shakthi Kannan
    * [mbuf] #guile #scheme ##linux-india #stumpwm #guix #dgplug #lisp #emacs 
    * [mbuf] kornbluth.freenode.net :Frankfurt, Germany
    * [mbuf] is connecting from *@123.123.123.123 123.123.123.123
    * [mbuf] idle 00:41:52, signon: Thu Sep  3 20:36:52
    * [mbuf] is logged in as mbuf
    * [mbuf] End of WHOIS list.
  5. /msg nickname is used to send a private message to a nickname and to start a private conversation.

  6. /help provides useful help on basic IRC commands. These commands are:

      ADDBUTTON ALLCHAN   ALLCHANL  ALLSERV   AWAY
      BACK      BAN       CHANOPT   CHARSET   CLEAR
      CLOSE     COUNTRY   CTCP      CYCLE     DCC
      DEBUG     DEHOP     DELBUTTON DEOP      DEVOICE
      DISCON    DNS       ECHO      EXEC      EXECCONT
      EXECKILL  EXECSTOP  EXECWRITE FLUSHQ    GATE
      GETFILE   GETINT    GETSTR    GHOST     GUI
      HELP      HOP       ID        IGNORE    INVITE
      JOIN      KICK      KICKBAN   KILLALL   LAGCHECK
      LASTLOG   LIST      LOAD      MDEHOP    MDEOP
      ME        MENU      MKICK     MODE      MOP
      MSG       NAMES     NCTCP     NEWSERVER NICK
      NOTICE    NOTIFY    OP        PART      PING
      QUERY     QUIT      QUOTE     RECONNECT RECV
      SAY       SEND      SERVCHAN  SERVER    SET
      SETCURSOR SETTAB    SETTEXT   SPLAY     TOPIC
      TRAY      UNBAN     UNIGNORE  UNLOAD    URL
      USELECT   USERLIST  VOICE     WALLCHAN  WALLCHOP
    
    User defined commands:
    
      ACTION    AME       ANICK     AMSG      BANLIST
      CHAT      DIALOG    DMSG      EXIT      GREP
      J         KILL      LEAVE     M         ONOTICE
      RAW       SERVHELP  SPING     SQUERY    SSLSERVER
      SV        UMODE     UPTIME    VER       VERSION
      WALLOPS   WII       
    
    Plugin defined commands:
    
      UNLOAD    UNLOAD    LOAD      PY        LOAD
      RELOADALL SOURCE    TCL       RELOADALL UNLOADALL
      PL_RELOAD RELOAD    UNLOAD    LOAD      TIMER
    
    
    Type /HELP <command> for more information, or /HELP -l
  7. /quit is used to disconnect and exit from IRC.

  8. /join #channel - is used to join a channel. For example: /join #guix.

  9. /nick newnickname changes your nick to newnickname. Suppose, you wish to move away from the computer, you can change your nick to nick|away, or nick|phone.

  10. /part is used to leave a channel.

Using rcirc

If you are using a recent GNU/Linux distribution, you should already have rcirc as part of GNU Emacs. You can simply start it by typing M-x rcirc from inside Emacs. The ‘M’ key represents the Meta key, which is usually mapped to the ‘Alt’ key. After rcirc connects to the IRC server, you can use /nick to change your nickname, register (only the first time) your nick, identify yourself, join channels, and start chatting! Since everything is a buffer in GNU Emacs, each channel is a separate buffer. For example, #emacs@irc.freenode.net is the #emacs IRC channel. All your basic buffer navigation commands will work just like they would on a file!

Some basic rcirc commands

The rcirc commands for the above mentioned IRC commands are given in the following table:

IRC rcirc
/list rcirc-cmd-list
/msg NickServ rcirc-authenticate
/me rcirc-cmd-me
/whois rcirc-cmd-whois
/msg nick rcirc-cmd-msg
/help /help
/quit rcirc-cmd-quit
/join rcirc-cmd-join
/nick rcirc-cmd-nick
/part rcirc-cmd-part

~/.emacs

GNU Emacs is an extensible editor. There are a number of locations where Emacs checks for custom configurations before launching the editor. These are: ~/.emacs, /.emacs.el, and/.emacs.d/init.el. The start-up files can be customised, and their locations can also be changed. There are a number of ways to organize, and manage your Emacs configuration. Until we get to learn Emacs lisp, and customize Emacs as a project, we shall use ~/.emacs for all our user-specific customizations.

rcirc customization

Create a ~/.emacs.d/etc folder in your $HOME directory and an Elisp file called init-rcirc.el in it. It should contain the following (change nick, user-name and full-name to suit your needs):

;; Default user.
(setq rcirc-default-nick "shaks")
(setq rcirc-default-user-name "shakthimaan")
(setq rcirc-default-full-name "Shakthi Kannan")

;; Channels to join at startup.
(setq rcirc-server-alist
      '(("irc.freenode.net" :channels ("##linux-india" "#dgplug" "#rcirc" "#emacs"))))

The above is an example of Emacs Lisp code. Comments begin with two semi-colons. The setq construct sets the second argument value to the first argument, which is a quoted symbol. For example, the symbol ’rcirc-default-nick is set to “shaks”. The rcirc-server-alist defines the initial list of channels to login at startup.

You can now start GNU Emacs from the GNOME terminal using the following command:

$ emacs -q -l ~/.emacs.d/etc/init-rcirc.el

You will then automatically connect to the four IRC channels.

How to use IRC

People join IRC channels to have their doubts regarding free and open source software clarified. Sometimes, off-topic discussions also happen. It is like live technical support, but has a social context to it. Whenever you are connected online, you must be logged in to IRC. You can have discussions in the channel, or in private, if the other party agrees. It is a good place to learn a lot about free and open source software, and you are bound to make a lot of friends. Since people from all over the world participate, which means they are online in different time zones, some channels log the discussions for future reference. As always, before asking a question, it is important for you to do your homework first. Take sufficient time and put in an effort to debug and identify the problem to the best of your ability.

Some users in the channel may ask you for more information before being able to provide you with any assistance. So, be prepared to provide all the information necessary about the bug or error when you seek help. Sometimes, people might be logged in the channel, but, they might be away from the computer. So, even if you don’t get a response, be patient; come back later and ask again.

You should not paste more than four continuous lines of text in the channel, as it will ‘flood’ the screen for everyone else. Instead, use an external paste service like gist.github.com, or fpaste.org. These services will provide a shortened URL that you can pass around in the channel. Whoever is interested in helping you will view the contents from the link. If you enter text in the channel, it means that it is addressed to everyone in the channel. If you wish to say something to a specific user, mention their nickname first, and then type in the text.

Most IRC client software provide you with panels that list the channels that you are logged in, and show the list of users. If someone mentions your nickname in a channel, then the corresponding channel will change colour or representation to indicate that there is a message for you. A few users are channel operators (or moderators) and they have special privileges. They are similar to ‘root’ users in a system, and they exist to keep the signal-to-noise ratio to a minimum, and keep a vigil on the channel.

An IRC bot is a client software that connects to the IRC server as a user, but can respond to commands. It can thus be programmed to provide many services in a channel. You can customize existing bots, or write your own. Examples of IRC bots are Cerberus, Gambot and irccd. Cinch is an example of an IRC bot building framework written in Ruby. Bots can be used during an IRC meeting session to keep track of user questions. They can evaluate programming language constructs and return meaningful errors to newbies in the channel. They can be used to send a notification to the channel if a project test build fails, or when a new bug has been filed. The possibilities are endless.

IRC meeting protocol

A free and open source software project will have a dedicated IRC channel where the project members will meet to have discussions. Meetings can be scheduled, and can happen in different time zones depending on where the users are located. There is a protocol and etiquette to be followed during such meetings. The speaker or moderator should not be interrupted during the session.

If you wish to ask a question, type ’?’ and wait. When the speaker has finished and feels that you can type in your text, you will be asked to do so. After you have finished typing your content, end with ‘EOF’. Similarly, if you need to speak during the session, type ’!’, and wait. You can give your consent or dissent to statements made in the channel using +1 or -1, respectively.

You are encouraged to read the rcirc manual and customize rcirc to your needs. If you have made it this far, do connect to irc.freenode.net, and feel free to say ‘Hi’ to me. I am ‘mbuf’ on irc.freenode.net. A screenshot of an rcirc session is shown below:

19:35 <mbuf>    http://ecb.sourceforge.net/screenshots/index.html
19:35 ***       Arpita QUIT Client Quit
19:36 <rtnpro>  !
19:36 <mbuf>    rtnpro, shoot!
19:37 <rtnpro>  How do we get the emacs code browser?
19:37 <rtnpro>  <EOF>
19:37 <mbuf>    rtnpro, 1. Need to install "ecb" from your distro package manager
                2. you could have searched this on the Internet :)
19:38 <rtnpro>  It is not in my distro
19:38 <sumitc>  !
19:38 <mbuf>    rtnpro, and which distro are you using?
19:38 <rtnpro>  Its Fedora 9
19:39 <rtnpro>  I have got emacs but not emacs code browser
19:39 <mbuf>    rtnpro, you can always install from source
19:39 <techno_freak>    rtnpro, http://ecb.sourceforge.net/downloads.html
19:39 ***       khushbu QUIT Ping timeout: 244 seconds
19:39 <rtnpro>  ok
19:39 <mbuf>    sumitc, shoot!
19:39 <sumitc>  what is a tag-file?
19:40 <rtnpro>  What factors should decide the choice of our editor?
19:40 <mbuf>    rtnpro, wait!
19:40 ***       pushkal JOIN
19:40 <mbuf>    sumitc, the TAGS file contains the details of the reference count,
                and locations of variables/functions et. al.
19:41 <sumitc>  So, a tag file is always associated with a specific file?
19:41 <mbuf>    sumitc, no, it can have information of files in a directory
19:41 <sumitc>  ok
19:41 <sumitc>  <eof>
19:42 <mbuf>    sumitc, think of it as a database that answers all your queries
               regarding code references

References

  1. Freenode. https://freenode.net/

  2. rcirc manual. https://www.gnu.org/software/emacs/manual/html_mono/rcirc.html

  3. IRC Help. http://www.irchelp.org/

  4. Fedora project - How to use IRC. https://fedoraproject.org/wiki/How_to_use_IRC

[Published in Open Source For You (OSFY) magazine, September 2015 edition.]

Continuing with our series on Emacs, in this article, let’s explore basic buffer navigation, formatting, and help commands.

Buffers

You can view a list of buffers in GNU Emacs using the C-x C-b command, and can switch to a buffer using the C-x b shortcut. The default list of buffers as seen in GNU Emacs 24.3.1 on Ubuntu 14.10 is shown below:

CRM Buffer                  Size Mode            File
.   *scratch*               191 Lisp Interaction 
*   *Messages*              485 Fundamental
%   *Completions*           264 Completion List
%   *Help*                 1855 Help

In order to kill (close) a buffer, you can use C-x k key combination. If you have multiple buffers that you would like to close, you can use M-x kill-some-buffers command, which will prompt you for a confirmation for every buffer. You can also rename a buffer using the M-x rename-buffer command. If you would like to save the buffer contents to a file, you can use C-x s, and it will prompt you with many options before proceeding to save the file. You can also save a modified buffer using C-x C-s.

Navigation

It is important to learn how to use the Emacs navigation commands instead of the arrow keys. When you learn to use macros, you will find it useful to automate steps. A summary of the basic navigation shortcuts are given in the following table:

Command Meaning
C-f Move forward one character
C-b Move backward one character
M-f Move forward one word
M-b Move back one word
C-p Move to previous line
C-n Move to next line
C-a Move to beginning of line
C-e Move to end of line

You can scroll down a buffer using C-v, and scroll up using M-v shortcut. If you would like to move to the beginning of a file, use M-< command. You can move to the end of a file using M-> shortcut.

In order to move the cursor to a specific line, use M-x goto-line or M-g g. If you would like to position the contents where the cursor is located to the center of the screen, you can use C-l. If you want to repeat a command ‘n’ (‘n’ is an integer) number of times, you can use M-n following by the command. You can also use C-u n, but, if you don’t specify ‘n’ here, the next command, by default, will be executed four times.

Clipboard

You can delete a character where the cursor is located using C-d. To remove a word, you can use M-d. You can remove text from the cursor position till the end of line using C-k. The cut text goes into a kill ring buffer. You can paste whatever you have removed using C-y. You can cycle through the text that has been removed by repeatedly using M-y after C-y. Consider the text:

When you have a question about Emacs, the Emacs manual is often the
best starting point.

If you place the cursor before “about” in the above sentence, and press ‘M-d’, it will remove " about“. You can now see the value of the kill-ring using C-h v (describe variable), and it will prompt you with the message”Describe variable:“, where you can enter kill-ring. You will see an entry for” about“. Now, place the cursor after”question" and press ‘M-d’ again, and “Emacs” will be removed. If you again check the kill-ring value, you will see an entry for " Emacs".

You can now move the cursor to any location in the buffer, and press ‘C-y’, and it will paste “Emacs”. If you then press M-y, it will paste “about” which was the text that was cut before “Emacs”. This way, you can cycle through the kill ring buffer.

To select a region of text, you first need to mark the starting position with the cursor using C-space. You can then use any of the navigation commands to highlight the text that you want. You can either cut the marked region using C-w, or copy it using M-w. This will get copied to the kill-ring. You can then paste the text to any buffer. You can also select the entire buffer using C-x h.

Formatting text

It is good to have text within 80 columns and 25 rows (80x25), not just for readability, but, to avoid making any assumptions about the users’ screen size. If you have a really long sentence or paragraph, you can reformat it using the M-q shortcut from anywhere in the sentence or paragraph.

You can transpose two characters using C-t shortcut. For example, if you have misspelt “emacs” as “emcas”, you can place the cursor on ‘a’ and type C-t to swap the letters ‘a’ and ‘c’ to get “emacs”. You can transpose two words using the M-t command. For example, if you have the phrase “morning good” in a buffer, and you type M-t anywhere on the word “morning”, it will swap the words to give you “good morning”. You can also swap lines and sentences using C-x C-t and M-x transpose-sentences commands respectively.

If you would like to capitalize the first character of a word in upper case, you need to place the cursor on it, and key in M-c to make the first letter capital. You can turn all those letters from the cursor location to the rest of the word into capitals using M-u command. For example, if you have the word “sentence”, and you place the cursor on the letter ’t’, and you key in M-u the result will be “senTENCE”. To make all the characters lower case, you can use M-l. You can turn the entire previous word into capital letters using the M - M-c shortcut. Similarily, to make the previous words either upper case and lower case, you can use M - M-u and M - M-l commands, respectively.

Info

You can read manuals written in Info format inside GNU Emacs. When you type M-x info, it opens up a menu with the built-in documentation available from ‘Info’ files. The Info mode is mostly used to read hypertext manuals. You can use Tab and Shift-tab to navigate through the available links. Since the contents are in a buffer, all the basic navigation commands are applicable. To exit from Info, you can use q (quit).

You can press the return key on a link to open its contents. For example, when you click on “Emacs FAQ” it opens “The GNU Emacs FAQ” contents. The following are the sections listed in its buffer:

* Menu
* FAQ notation
* General questions
* Getting help
* Status of Emacs
* Common requests
* Bugs and problems
* Compiling and installing Emacs
* Finding Emacs and related packages
* Key bindings
* Alternate character sets
* Mail and news
* Concept index

Suppose, you hit the return key on “FAQ notation”, it will open its contents with the title “1 FAQ notation”. If you want to cycle through the headings at this level, you can press n (next topic) and p (previous topic). If you want to move one level up, to the “The GNU Emacs FAQ”, you can use the u shortcut.

If you simply want to move to the topmost node (“The GNU Emacs FAQ”), you can use the t shortcut. For example, if you were in the section “3.1 I’m just starting Emacs; how do I do basic editing?” and you pressed ’t’, it will take you to the topmost section - “The GNU Emacs FAQ”.

You can move to the previous node that you visited using the l shortcut. You can also explicitly specify the node you want to move to using the g shortcut. For example, if you are in the “1 FAQ notation” page, and you pressed g, it will then prompt you with the message “Go to node:”. You can then type “Extended” and hit TAB to auto-complete to “Extended commands”, which is one of the links in the page.

You are encouraged to try the built-in info tutorial available by pressing h in the “info” buffer.

Help

You can read manual pages inside GNU Emacs using M-x man command followed by the return key. Suppose, you try this command from the scratch buffer, it will then prompt you with the message “Manual entry:”. You can then input ls (list directory contents), and it will open a new buffer and window titled “Man ls” with the contents of the manual entry for the ls command.

From the scratch buffer, you can scroll forward the other window (‘Man ls’) using C-M-v (Ctrl-Alt-v), and scroll back using C-M-shift-v (Ctrl-Alt-Shift-v).

You can list all the available, active key-bindings using the C-h b command.

The C-h f shortcut is used to provide documentation for a function available in your installed version of GNU Emacs. For example, if you use C-h f, it will prompt you with the message “Describe function:”. If you input scroll-other-window, it will open a new buffer and window titled “Help” with the description and usage of the function. The output as seen in GNU Emacs 24.3.1 is shown below:

scroll-other-window is an interactive built-in function in `C source
code'.

It is bound to <M-next>, C-M-v, ESC <next>.

(scroll-other-window &optional ARG)

Scroll next window upward ARG lines; or near full screen if no ARG.
A near full screen is `next-screen-context-lines' less than a full screen.
The next window is the one below the current one; or the one at the top
if the current one is at the bottom.  Negative ARG means scroll downward.
If ARG is the atom `-', scroll downward by nearly full screen.
When calling from a program, supply as argument a number, nil, or `-'.

If `other-window-scroll-buffer' is non-nil, scroll the window
showing that buffer, popping the buffer up if necessary.
If in the minibuffer, `minibuffer-scroll-window' if non-nil
specifies the window to scroll.  This takes precedence over
`other-window-scroll-buffer'.

[back]

On the other hand, if you know the command, and would like to get help related to it, you can use the C-h k command. This will prompt you with the message “Describe key (or click or menu item):”. You must key in the actual shortcut, for example, C-h b, and it will update the buffer “Help”, and open it in a new window, showing the description for the shortcut as shown below:

C-h b runs the command describe-bindings, which is an interactive
compiled Lisp function.

It is bound to C-h b, <f1> b, <help> b, <menu-bar> <help-menu>
<describe> <list-keybindings>.

(describe-bindings &optional PREFIX BUFFER)

Show a list of all defined keys, and their definitions.
We put that list in a buffer, and display the buffer.

The optional argument PREFIX, if non-nil, should be a key sequence;
then we display only bindings that start with that prefix.
The optional argument BUFFER specifies which buffer's bindings
to display (default, the current buffer).  BUFFER can be a buffer
or a buffer name.

[back]

You are encouraged to try out all the commands so that you understand what actions they perform. When you use GNU Emacs as your main text editor, you will be able to regularly practice these commands, and master them.

[Published in Open Source For You (OSFY) magazine, August 2015 edition.]

Introduction

Dired is a directory editor in GNU Emacs. It opens a buffer containing a list of directories and files to operate on. It is a ‘read-only’ mode and hence you cannot input any text. This article explores some of the basic commands that can be used in Dired mode.

Let us first create a sample directory, sub-directories and files that we can use for our demonstration. Open a terminal and execute the following commands:

$ mkdir /tmp/test/network -p
$ mkdir /tmp/test/kernel -p

$ date > /tmp/test/date.txt
$ locale > /tmp/test/locale.txt

$ cat /etc/resolv.conf > /tmp/test/network/resolv.conf
$ ifconfig > /tmp/test/network/ifconfig.output

$ dmesg > /tmp/test/kernel/dmesg.txt

In general, filename extensions don’t have any meaning on *nix systems. Some applications do check the filename extension before using them. The extension is only for the benefit of the user, and hence you can have a filename without any extension on *nix.

Invocation

After opening GNU Emacs, you can enter Dired mode using M-x dired. It will prompt you for the directory to be opened in the buffer where you can input ‘/tmp/test’. You will see a buffer with the following contents:

/tmp/test:
total used in directory 24 available 146721468
drwxrwxr-x  4 shakthi shakthi 4096 Jul  3 11:36 .
drwxrwxrwt 12 root    root    4096 Jul  3 11:38 ..
-rw-rw-r--  1 shakthi shakthi   29 Jul  3 11:36 date.txt
drwxrwxr-x  2 shakthi shakthi 4096 Jul  3 11:36 kernel
-rw-rw-r--  1 shakthi shakthi  270 Jul  3 11:36 locale.txt
drwxrwxr-x  2 shakthi shakthi 4096 Jul  3 11:36 network

The contents of the buffer are similar to the ls -al output as observed in the terminal. The files were created by user ‘shakthi’, and this is indicated in the owner and group fields. The ’.’ entry represents the current ‘/tmp/test’ directory. The ’..’ listing represents the parent directory, which is ‘/tmp’ and owned by the ‘root’ user.

Exit

You can exit from Dired mode by simply pressing q (quit-window) in the Dired buffer.

Navigation

You can move to a previous line or the next line in the Dired buffer using the p and n keys, respectively. If you wish to move the cursor to the previous and next directories, you can use the ‘<’ and ‘>’ keys. If the cursor is at ‘kernel’ directory as shown below …

/tmp/test:
total used in directory 24 available 146721468
drwxrwxr-x  4 shakthi shakthi 4096 Jul  3 11:36 .
drwxrwxrwt 12 root    root    4096 Jul  3 11:38 ..
-rw-rw-r--  1 shakthi shakthi   29 Jul  3 11:36 date.txt
drwxrwxr-x  2 shakthi shakthi 4096 Jul  3 11:36 kernel      <-- CURSOR
-rw-rw-r--  1 shakthi shakthi  270 Jul  3 11:36 locale.txt
drwxrwxr-x  2 shakthi shakthi 4096 Jul  3 11:36 network

… then, when you press ‘>’, the cursor will move to ‘network’, which is the next directory in the buffer.

/tmp/test:
total used in directory 24 available 146721468
drwxrwxr-x  4 shakthi shakthi 4096 Jul  3 11:36 .
drwxrwxrwt 12 root    root    4096 Jul  3 11:38 ..
-rw-rw-r--  1 shakthi shakthi   29 Jul  3 11:36 date.txt
drwxrwxr-x  2 shakthi shakthi 4096 Jul  3 11:36 kernel      
-rw-rw-r--  1 shakthi shakthi  270 Jul  3 11:36 locale.txt
drwxrwxr-x  2 shakthi shakthi 4096 Jul  3 11:36 network     <-- CURSOR

You can move to the parent directory using the ‘^’ key. To enter into a directory, move to its listing in the Dired buffer, and simply hit the return key.

Viewing files

To view a file, you can place the cursor on its entry and use the f or v key, or simply hit the return key. This will open the file in a new buffer (view). To return to the Dired buffer, you can type C-x b and the minibuffer will prompt you with the message “Switch to buffer (…)”. If you then press TAB, it will open a new Completions buffer that will list the available buffers. You can also type the entire ‘test’ buffer name, or type it partially and hit TAB for auto-completion, and hit the return key to get to the ‘test’ buffer. If you wish to close a buffer, you can use C-k to kill it. This will only close the buffer, but, the file will still exist! You can use the ‘+’ key to create a new sub-directory.

Marking and unmarking files

Dired mode allows you to operate on multiple files and directories. In order to run commands on them, you need to first select the files. The m key can be used to mark a file or directory for subsequent operations. For example, pressing ’m’ on the date.txt entry will mark it, and this is indicated by an asterisk in front of the line, as shown below:

  /tmp/test:
  total used in directory 24 available 146933380
  drwxrwxr-x  4 shakthi shakthi 4096 Jul  5 09:53 .
  drwxrwxrwt 10 root    root    4096 Jul  5 09:46 ..
* -rw-rw-r--  1 shakthi shakthi   29 Jul  5 09:46 date.txt
  drwxrwxr-x  2 shakthi shakthi 4096 Jul  5 09:46 kernel
  -rw-rw-r--  1 shakthi shakthi  270 Jul  5 09:46 locale.txt
  drwxrwxr-x  2 shakthi shakthi 4096 Jul  5 09:46 network

You can unmark the above selection using the u key. This is applicable for both files and directories. To undo a selection, you can also use M-del (M and the Delete key). Suppose, you wish to mark all the directories in the Dired buffer, you can use ‘* /’ key combination. The resultant Dired buffer is shown below:

  /tmp/test:
  total used in directory 24 available 146933432
  drwxrwxr-x  4 shakthi shakthi 4096 Jul  5 09:53 .
  drwxrwxrwt 10 root    root    4096 Jul  5 09:46 ..
  -rw-rw-r--  1 shakthi shakthi   29 Jul  5 09:46 date.txt
* drwxrwxr-x  2 shakthi shakthi 4096 Jul  5 09:46 kernel
  -rw-rw-r--  1 shakthi shakthi  270 Jul  5 09:46 locale.txt
* drwxrwxr-x  2 shakthi shakthi 4096 Jul  5 09:46 network

If you want to invert the selection, use t. The buffer will look like the following:

  /tmp/test:
  total used in directory 24 available 146933432
  drwxrwxr-x  4 shakthi shakthi 4096 Jul  5 09:53 .
  drwxrwxrwt 10 root    root    4096 Jul  5 09:46 ..
* -rw-rw-r--  1 shakthi shakthi   29 Jul  5 09:46 date.txt
  drwxrwxr-x  2 shakthi shakthi 4096 Jul  5 09:46 kernel
* -rw-rw-r--  1 shakthi shakthi  270 Jul  5 09:46 locale.txt
  drwxrwxr-x  2 shakthi shakthi 4096 Jul  5 09:46 network

To mark all the files, you can use ‘* s’ key combination.

Modifying the buffer

After you have selected files, you can remove them from the listing by pressing the k key. This does not delete the files! You can always press g to refresh the Dired buffer contents. You can change the ‘ls’ command line options used in the Dired listing using the C-u s key combination.

Actions

You can perform a number of operations on the marked files and directories. To copy a file, you can press C on a file, and it will prompt you in the minibuffer regarding where you would like to create the new copy. You can use R to rename a file. If you would like to delete a file, you can press D. You can change the mode of a file with the M command. You can compress or uncompress a file with the Z command.

A regular expression search can be done on selected files with the A command. You can create a symbolic link file with the S key. You can run a shell command on marked files using the ’!’ command. For example, if you want to perform a word count (wc) on the date.txt file, you can press ’!’ on the date.txt entry listing, and it will prompt you in the minibuffer with “! on date.txt:” message. If you input ‘wc’, the results will again be shown in the minibuffer. On my system, it returned ‘1 6 29 date.txt’. You can also run asynchronous shell commands on files using the ‘&’ command, and a new buffer will be opened that will contain the results.

Deletion

You can mark files for deletion using the d command. After selecting the files or directories, if you press x, the respective actions will be performed on the files. To demonstrate an example, let us use ‘d’ on the ‘kernel’ directory to mark it for deletion. The letter ‘D’ will be added to its entry as shown below:

  /tmp/test:
  total used in directory 24 available 146933432
  drwxrwxr-x  4 shakthi shakthi 4096 Jul  5 09:53 .
  drwxrwxrwt 10 root    root    4096 Jul  5 09:46 ..
  -rw-rw-r--  1 shakthi shakthi   29 Jul  5 09:46 date.txt
D drwxrwxr-x  2 shakthi shakthi 4096 Jul  5 09:46 kernel
  -rw-rw-r--  1 shakthi shakthi  270 Jul  5 09:46 locale.txt
  drwxrwxr-x  2 shakthi shakthi 4096 Jul  5 09:46 network

When you press ‘x’, you will be prompted to confirm deletion with the message “Delete kernel (yes or no)”. When you input “yes” and hit the return key, the directory contents will be removed. GNU Emacs will perform auto-save on files, and also create backup files ending with ‘~’. You can mark such files using the ‘~’ command.

Regex

You can use ‘% d’ to select files based on a regular expression (regex) to mark for deletion. To simply mark files based on regex, you can use ‘% m’ shortcut. To rename the marked files, you can use ‘% R’.

Find

You can search for files matching a pattern using the M-x find-name-dired command. Suppose you wish to find all the .txt files in our original /tmp/test directory, you can use ‘M-x find-name-dired’ in the Dired buffer. It will prompt you in the minibuffer with the following message “Find-name (directory): /tmp/test”. After you press the return key, it will ask for the regex with the message “Find-name (filename wildcard):”. If you input ‘*.txt’, it will return the following contents:

/tmp/test/:
find . \( -name \*.txt \) -ls
6826601    4 -rw-rw-r--   1 shakthi  shakthi        29 Jul  5 09:46 date.txt
6826616   64 -rw-rw-r--   1 shakthi  shakthi     63050 Jul  5 09:46 kernel/dmesg.txt
6826602    4 -rw-rw-r--   1 shakthi  shakthi       270 Jul  5 09:46 locale.txt

find finished at Sun Jul  5 10:27:02

You can also find files based on regex patterns that exist in the file contents using the M-x find-grep-dired. Suppose, you wish to find the files that have the text ‘eth’ in them, you can use ‘M-x find-grep-dired’, which will prompt you in the minibuffer with the message “Find-grep (directory): /tmp/test”. After you press the return key, it will prompt you for the regex with the message “Find-grep (grep regexp):”, where you can input ‘eth’ and this will return the following results:

/tmp/test/:
find . \( -type f -exec grep -q -e eth \{\} \; \) -ls
6826605    4 -rw-rw-r--   1 shakthi  shakthi      1620 Jul  5 09:46 network/ifconfig.output
6826616   64 -rw-rw-r--   1 shakthi  shakthi     63050 Jul  5 09:46 kernel/dmesg.txt

find finished at Sun Jul  5 10:31:27

You can also execute a find operation on marked files using M-x find-dired.

Help

To open the help menu for Dired, you can use the h shortcut key. To see the Dired log messages, you can press the ? key.

You may refer to a quick reference card at https://www.gnu.org/software/emacs/refcards/pdf/dired-ref.pdf, and try out more Dired commands.

[Published in Open Source For You (OSFY) magazine, July 2015 edition.]

Introduction

GNU Emacs is a very popular text editor written in C and Emacs Lisp. It can be run on many platforms, and can be easily customised and extended for user needs. It was created by Richard Stallman, the founder of the GNU project. This article is the first in a series on how to use it. There are a number of tutorials available for GNU Emacs; this series of articles provide one approach to understanding and learning the software. You are encouraged to refer to the official GNU Emacs reference manual for more information, and it supersedes everything. Here are a few interesting quotes on Emacs.

“I use Emacs, which might be thought of as a thermonuclear word processor” – Neal Stephenson (In the Beginning… Was the Command Line)

“Emacs is undoubtedly the most powerful programmer’s editor in existence. It’s a big, feature-laden program with a great deal of flexibility and customizability. … Emacs has an entire programming language inside it that can be used to write arbitrarily powerful editor functions.” – Eric S. Raymond (from ‘The Art of UNIX Programming’)

“Personally, I feel inspired whenever I open Emacs. Like a craftsman entering his workshop, I feel a realm of possibility open before me. I feel the comfort of an environment that has evolved over time to fit me perfectly - an assortment of packages and keybindings which help me bring ideas to life day after day.” – Daniel Higginbotham (in ‘Clojure for the Brave and True’)

“EMACS could not have been reached by a process of careful design, because such processes arrive only at goals which are visible at the outset, and whose desirability is established on the bottom line at the outset. Neither I nor anyone else visualized an extensible editor until I had made one, nor appreciated its value until he had experienced it. EMACS exists because I felt free to make individually useful small improvements on a path whose end was not in sight.” – Richard Stallman

Installation

You can use your favourite GNU/Linux distribution package manager to install GNU Emacs. On Debian/Ubuntu, you can install with the following command:

$ sudo apt-get install emacs

On Fedora, you can use the Yum package manager as shown below:

$ sudo yum install emacs

The emerge tool can install GNU Emacs on Gentoo, as follows:

# emerge --ask app-editors/emacs

On Arch, the Pacman software can help you in installing GNU Emacs:

$ sudo pacman -S emacs

Use the Zypper package manager in SUSE as shown below:

$ sudo zypper install emacs

Startup

If you use the Unity interface, you can search for emacs in the Dash, and it will show the Emacs icon that you can click to open the editor. This is illustrated in the following figure.

Unity Emacs search image

On the Metacity interface or any other desktop environment with a panel, you can open GNU Emacs from “Applications” -> “Accessories” -> “Emacs” as shown below:

Metacity Emacs

You can also open the editor from the terminal by simply typing ‘emacs’ and hitting the return key.

$ emacs

The version that I have used for this article is GNU Emacs 24.3.1.

Exit

To exit from GNU Emacs, you need to use C-c C-q, where ‘C’ stands for the Control key. You can also use your mouse and close the editor by clicking on the ‘x’, but, GNU Emacs was designed to be completely usable with a keyboard, and I am going to encourage you to only use the keyboard shortcuts.

Concepts

While you can work using GUI editors, I am going to teach you to work entirely on the keyboard to experience the power of shortcuts in GNU Emacs. You can disconnect the mouse and your touchpad when working on GNU Emacs. Just as an exercise, I’d encourage you to remove your mouse completely and see how you can work with a computer for one day. You will realise that a lot of user interfaces are heavily dependent and designed for mouse interactions! By only using the keyboard, you can be blazingly fast and productive.

The keyboard shortcuts in GNU Emacs may seem to involve many keys. But, please bear with me on this, because the way the shortcuts are designed, you will be able to remember them easily. As you practice, you will gain insight into how consistently they have been defined.

GNU Emacs is a ‘stateless’ editor for the most part. By ‘stateless’, I mean that there are no specific state transitions that need to happen before you can use the commands. There does exist the concept of modes. When you open GNU Emacs, you will see menus, buffer and a mode line as illustrated in the following figure:

Emacs default screen

As mentioned earlier, we will not be clicking on the menus or icons with a mouse, but only use keyboard shortcuts. Everything is a buffer in GNU Emacs. Each buffer can have one major mode and one or more minor modes. The mode determines the keyboard shortcuts that are applicable primarily on the buffer. Examples of major modes are given below:

Mode Description
Text mode Writing text
HTML mode Writing HTML
cc mode Writing C, C++ and C-like programs
Dired mode Handling files and directories
Shell mode Working with shell
LaTeX mode Formatting TeX and LaTeX files
Picture mode Creating ASCII art
Outline mode Writing outlines
SQL mode Interacting with SQL databases
Lisp mode Writing Lisp programs

The mode line exists below the buffer and it gives you a lot of information on the status of the buffer, such as what modes are active and other useful information. Below the mode line is the mini-buffer, where any commands you issue are indicated and prompts for user input are shown. This is the overall view of the default GNU Emacs interface.

In today’s user interface applications, an application is treated as a window on a desktop. But, when you open GNU Emacs, you are actually opening a frame. A frame can be split it into many windows. Everything is a buffer in GNU Emacs. So, you can have many frames of GNU Emacs, and inside each you can have one or more windows containing buffers (or files).

Features

Although GNU Emacs was designed primarily for text editing, it can do a lot more. ‘A lot’ is probably a highly simplified term. It has got support for syntax highlighting for a large number of programming languages. You can also generate code snippets from templates using the yasnippet package. You can also enter markup or markdown text. There is support for indentation of text and programs depending on the programming languages and modes. Internationalization support is available, and you can use it to even enter text in Indian languages.

A number of configurations are available for setting it up for your development work, including automating tasks for compiling, executing, testing and deployment. When you become familiar with Emacs Lisp, you can implement your own modules. Since, it is Lisp, it is also easily extensible. You can write your own macros to perform repeated tasks. You can also query an inbuilt help system for information, shortcuts, tutorials and other documentation. You are not at all dependent on the Internet for information, and thus you can work offline too. Version control support is available for many centralized (cvs, svn) and decentralized systems (Git, Hg).

org-mode is a very popular mode for managing your notes. You can use it for planning your day-to-day activities. GNU Emacs can be used as publishing software. You can create wikis, blogs and publish books using it. This article is written using org-mode. Spell-checking modules are also available for your documentation needs. It is also possible to export plain text into a number of formats (PDF, HTML etc.).

A number of Emacs lisp packages are available for networking. You can use Gnus for checking your e-mails, and reading and writing to newsgroups. Emacs Relay Chat (ERC) can be used for connecting to Internet Relay Chat (IRC) channels. There are modules that support the Jabber protocol for communicating with chat servers. There is also support for viewing Web pages inside Emacs. There are a large number of Emacs modules available through package repositories such as MELPA (melpa.org), and Marmalada (marmalade-repo.org).

History

The first version of Emacs (macros) was written by Richard Stallman and Guy L. Steele, Jr. in 1976 for the TECO editor in MIT. It was written in a low-level language for the PDP-10 assembler. People were able to freely hack on the code, make improvements and share their changes. This was the original hacker culture that existed in MIT. Unfortunately, business entities started to make software proprietary and this hacker culture ceased to exist, especially in the MIT AI labs. Richard Stallman wanted to revive the hacker culture and started the GNU project. He wrote the second implementation of the editor entirely in C in 1984, and released it as the first program of the GNU project. Today, it is a very popular editor that is widely used, and has more than 570 contributors. The official web site for GNU Emacs is at http://www.gnu.org/software/emacs/.

[Published in Open Source For You (OSFY) magazine, June 2015 edition.]

In this final article in the Haskell series, we shall explore how to use it for web programming.

Scotty is a web framework written in Haskell, which is similar to Ruby’s Sinatra. You can install it on Ubuntu using the following commands:

$ sudo apt-get install cabal-install
$ cabal update
$ cabal install scotty

Let us write a simple `Hello, World!’ program using the Scotty framework:

-- hello-world.hs

{-# LANGUAGE OverloadedStrings #-}

import Web.Scotty

main :: IO ()
main = scotty 3000 $ do
  get "/" $ do
    html "Hello, World!"

You can compile and start the server from the terminal using the following command:

$ runghc hello-world.hs 
Setting phasers to stun... (port 3000) (ctrl-c to quit)

The service will run on port 3000, and you can open localhost:3000 in a browser to see the `Hello, World!’ text. You can then stop the service by pressing Control-c in the terminal. You can also use Curl to make a query to the server. Install and test it on Ubuntu as shown below:

$ sudo apt-get install curl

$ curl localhost:3000
Hello, World!

You can identify the user client that made the HTTP request to the server by returning the “User-Agent” header value as illustrated in the following example:

-- request-header.hs

{-# LANGUAGE OverloadedStrings #-}

import Web.Scotty

main :: IO ()
main = scotty 3000 $ do
  get "/agent" $ do
    agent <- header "User-Agent"
    maybe (raise "User-Agent header not found!") text agent

You can execute the above code in a terminal using the following command:

$ runghc request-header.hs  
Setting phasers to stun... (port 3000) (ctrl-c to quit)

If you open the URL localhost:3000/agent in the browser, it returns the following User-Agent information on Ubuntu 14.10 Mozilla/5.0 (X11; Linux x8664) AppleWebKit/537.36 (KHTML, like Gecko) Ubuntu Chromium/41.0.2272.76 Chrome/41.0.2272.76 Safari/537.36. The Curl version is returned for the same URL request as shown below:

$ curl localhost:3000/agent -v

 * Hostname was NOT found in DNS cache
 *   Trying 127.0.0.1...
 * Connected to localhost (127.0.0.1) port 3000 (#0)
 > GET /agent HTTP/1.1
 > User-Agent: curl/7.37.1
 > Host: localhost:3000
 > Accept: */*
 > 
 < HTTP/1.1 200 OK
 < Transfer-Encoding: chunked
 < Date: Wed, 29 Apr 2015 07:46:21 GMT
 * Server Warp/3.0.12.1 is not blacklisted
 < Server: Warp/3.0.12.1
 < Content-Type: text/plain; charset=utf-8
 < 
 * Connection #0 to host localhost left intact

 curl/7.37.1

You can also return different content types (HTML, text, JSON) based on the request. For example:

-- content-type.hs

{-# LANGUAGE OverloadedStrings #-}

import Web.Scotty as W
import Data.Monoid
import Data.Text
import Data.Aeson

main :: IO ()
main = scotty 3000 $ do
  get "/hello" $ do
    html $ mconcat ["<h1>", "Hello, World!", "</h1>"]

  get "/hello.txt" $ do
    text "Hello, World!"

  get "/hello.json" $ do
    W.json $ object ["text" .= ("Hello, World!" :: Text)]

You can start the above server in a terminal as follows:

$ runghc content-type.hs 
Setting phasers to stun... (port 3000) (ctrl-c to quit)

You can then open the three URLs listed above in a browser to see the different output. The respective outputs when used with Curl are shown below:

$ curl localhost:3000/hello
<h1>Hello, World!</h1>

$ curl localhost:3000/hello.txt
Hello, World!

$ curl localhost:3000/hello.json
{"text":"Hello, World!"}

You can also pass parameters in the URL when you make a request. The param function can be used to retrieve the parameters as indicated below:

-- params.hs

{-# LANGUAGE OverloadedStrings #-}

import Web.Scotty
import Data.Monoid

main :: IO ()
main = scotty 3000 $ do
  get "/user" $ do
    name <- param "name"
    html $ mconcat ["<h1>Hello ", name, "</h1>"]

You can start the above server using the runghc command:

$ runghc params.hs 
Setting phasers to stun... (port 3000) (ctrl-c to quit)

You can now try the URL requests with and without parameters. The observed outputs are shown below:

$ curl localhost:3000/user
<h1>500 Internal Server Error</h1>Param: name not found!

$ curl localhost:3000/user?name=Shakthi
<h1>Hello Shakthi</h1>

The Hspec testing framework can be used for integration testing the web application. Install the required dependencies as shown below:

$ cabal install happy hspec hspec-wai hspec-wai-json

The content type example has been updated to use Hspec, as illustrated below:

-- content-type-spec.hs

{-# LANGUAGE OverloadedStrings, QuasiQuotes #-}
module Main (main) where

import Data.Monoid
import Data.Text

import           Network.Wai (Application)
import qualified Web.Scotty as W
import           Data.Aeson (object, (.=))

import           Test.Hspec
import           Test.Hspec.Wai
import           Test.Hspec.Wai.JSON

main :: IO ()
main = hspec spec

app :: IO Application
app = W.scottyApp $ do
  W.get "/hello.txt" $ do
    W.text "Hello, World!"

  W.get "/hello" $ do
    W.html $ mconcat ["<h1>", "Hello, World!", "</h1>"]

  W.get "/hello.json" $ do
    W.json $ object ["text" .= ("Hello, World!" :: Text)]

spec :: Spec
spec = with app $ do
  describe "GET /" $ do
    it "responds with text" $ do
      get "/hello.txt" `shouldRespondWith` "Hello, World!"

    it "responds with HTML" $ do
      get "/hello" `shouldRespondWith` "<h1>Hello, World!</h1>"

    it "responds with JSON" $ do
      get "/hello.json" `shouldRespondWith` [json|{text: "Hello, World!"}|]

You can compile the above code as shown below:

$ ghc --make content-type-spec.hs     
Linking content-type-spec ...

The following output is observed when you run the above built test executable:

$ ./content-type-spec 

GET /
  responds with text
  responds with HTML
  responds with JSON

Finished in 0.0010 seconds
3 examples, 0 failures

Please refer to the hspec-wai webpage at https://github.com/hspec/hspec-wai for more information.

Template support is available through many Haskell packages. The use of the blaze-html package is demonstrated below. Install the package first using the following command:

$ cabal install blaze-html

Consider a simple web page with a header and three unordered lists. Using blaze-html, the template can be written in Haskell DSL as follows:

-- template.hs

{-# LANGUAGE OverloadedStrings #-}

import Web.Scotty as W
import Text.Blaze.Html5
import Text.Blaze.Html.Renderer.Text

main :: IO ()
main = scotty 3000 $ do
  get "/" $ do
    W.html . renderHtml $ do
      h1 "Haskell list"
      ul $ do
        li "http://haskell.org"
        li "http://learnyouahaskell.com/"
        li "http://book.realworldhaskell.org/"

You can compile the above code using GHC:

$ ghc --make template.hs 
Linking template ...

You can then execute the built executable, which starts the server as shown below:

$ ./template 
Setting phasers to stun... (port 3000) (ctrl-c to quit)

Opening a browser with URL localhost:3000 will render the expected HTML file. You can also verify the resultant HTML output using the Curl command as shown below:

$ curl localhost:3000
<h1>Haskell list</h1><ul><li>http://haskell.org</li><li>http://learnyouahaskell.com/</li><li>http://book.realworldhaskell.org/</li></ul>

It is good to separate the views from the actual application code. You can move the template content to a separate file as shown below:

-- Haskell.hs

{-# LANGUAGE OverloadedStrings #-}

module Haskell where

import Text.Blaze.Html5

render :: Html
render = do
  html $ do
    body $ do
      h1 "Haskell list"
      ul $ do
        li "http://haskell.org"
        li "http://learnyouahaskell.com/"
        li "http://book.realworldhaskell.org/"

The main application code is now simplified as shown below:

-- template-file.hs

{-# LANGUAGE OverloadedStrings #-}

import qualified Haskell
import Web.Scotty as W
import Text.Blaze.Html
import Text.Blaze.Html.Renderer.Text

blaze :: Text.Blaze.Html.Html -> ActionM ()
blaze = W.html . renderHtml

main :: IO ()
main = scotty 3000 $ do
  get "/" $ do
    blaze Haskell.render

You need to place both the source files (Haskell.hs and template-file.hs) in the same top-level directory, and you can then compile the template-file.hs file that will also compile the dependency Haskell.hs source file as shown below:

$ ghc --make template-file.hs 

[1 of 2] Compiling Haskell          ( Haskell.hs, Haskell.o )
[2 of 2] Compiling Main             ( template-file.hs, template-file.o )
Linking template-file ...

You can now run the server as follows:

$ ./template-file 
Setting phasers to stun... (port 3000) (ctrl-c to quit)

Executing template-file produces the same output as in the case of the template.hs example.

$ curl localhost:3000
<html><body><h1>Haskell list</h1><ul><li>http://haskell.org</li><li>http://learnyouahaskell.com/</li><li>http://book.realworldhaskell.org/</li></ul></body></html>

You can refer the Scotty wiki page at https://github.com/scotty-web/scotty/wiki for more information.

The clay package is a CSS preprocessor similar to LESS and Sass. You can install it using the following Cabal command:

    $ cabal install clay

Let us consider a simple CSS example to generate a list of fonts to be used in the body section of a HTML page. The corresponding Clay Haskell embedded DSL looks like the following:

-- clay-simple.hs

{-# LANGUAGE OverloadedStrings #-}

import Clay

main :: IO ()
main = putCss exampleStylesheet

exampleStylesheet :: Css
exampleStylesheet = body ? fontFamily ["Baskerville", "Georgia", "Garamond", "Times"] [serif]

You can compile the above code as follows:

$ ghc --make clay-simple.hs 
[1 of 1] Compiling Main             ( clay-simple.hs, clay-simple.o )
Linking clay-simple ...

You can then execute clay-simple to generate the required CSS output as shown below:

$ ./clay-simple

body
{
  font-family : "Baskerville","Georgia","Garamond","Times", serif;
}

/* Generated with Clay, http://fvisser.nl/clay */

A more comprehensive example is shown below for the HTML pre tag:

-- clay-pre.hs

{-# LANGUAGE OverloadedStrings #-}

import Clay

main :: IO ()
main = putCss $
  pre ?
    do border dotted (pt 1) black
       whiteSpace (other "pre")
       fontSize (other "8pt")
       overflow (other "auto")
       padding (em 20) (em 0) (em 20) (em 0)

You can compile the above clay-pre.hs file as shown below:

$ ghc --make clay-pre.hs 
[1 of 1] Compiling Main             ( clay-pre.hs, clay-pre.o )
Linking clay-pre ...

Executing the above complied clay-pre binary produces the following output:

$ ./clay-pre 

pre
{
  border      : dotted 1pt rgb(0,0,0);
  white-space : pre;
  font-size   : 8pt;
  overflow    : auto;
  padding     : 20em 0em 20em 0em;
}

/* Generated with Clay, http://fvisser.nl/clay */

You can also add custom values using the Other type class or the fallback operator `-:’ to explicitly specify values. For example:

-- clay-custom.hs

{-# LANGUAGE OverloadedStrings #-}

import Clay

main :: IO ()
main = putCss $
  body ?
       do fontSize (other "11pt !important")
          "border" -: "0"

Compiling and executing the above code produces the following output:

$ ghc --make clay-custom.hs 
[1 of 1] Compiling Main             ( clay-custom.hs, clay-custom.o )
Linking clay-custom ...

$ ./clay-custom 

body
{
  font-size : 11pt !important;
  border    : 0;
}

/* Generated with Clay, http://fvisser.nl/clay */

You can explore more of clay from the official project homepage http://fvisser.nl/clay/.

A number of good books are available for further learning. I recommend the following books available online and in print:

  1. Bryan O’Sullivan, Don Stewart, and John Goerzen. (December 1, 2008). Real World Haskell (http://book.realworldhaskell.org/). O’Reilly.

  2. Miran Lipovaca. (April 21, 2011). Learn You a Haskell for Great Good! A Beginner’s Guide (http://learnyouahaskell.com/). No Starch Press.

The https://www.haskell.org website also has plenty of useful resources. You can also join the haskell-cafe@haskell.org and beginners@haskell.org mailing lists ( https://wiki.haskell.org/Mailing_lists ) for discussions. The folks in the #haskell channel on irc.freenode.net are also very helpful.

I hope you enjoyed learning Haskell through this series, as much as I did creating them. Please feel free to write to me (author at shakthimaan dot com) with any feedback or suggestions.

Happy Hacking!

[Published in Open Source For You (OSFY) magazine, May 2015 edition.]

In this article we shall explore network programming in Haskell.

Let us begin with a simple TCP (Transmission Control Protocol) client and server example. The network package provides a high-level interface for communication. You can install the same in Fedora, for example, using the following command:

$ sudo yum install ghc-network

Consider the following simple TCP client code:

-- tcp-client.hs

import Network
import System.IO

main :: IO ()
main = withSocketsDo $ do
         handle <- connectTo "localhost" (PortNumber 3001)
         hPutStr handle "Hello, world!"
         hClose handle

After importing the required libraries, the main function connects to a localhost server running on port 3001, sends a string “Hello, world!”, and closes the connection.

The connectTo function defined in the Network module accepts a hostname, port number and returns a handle that can be used to transfer or receive data.

The type signatures of the withSocketsdo and connectTo functions are as under:

ghci> :t withSocketsDo
withSocketsDo :: IO a -> IO a

ghci> :t connectTo
connectTo :: HostName -> PortID -> IO GHC.IO.Handle.Types.Handle

The simple TCP server code is illustrated below:

-- tcp-server.hs

import Network
import System.IO

main :: IO ()
main = withSocketsDo $ do
         sock <- listenOn $ PortNumber 3001
         putStrLn "Starting server ..."
         handleConnections sock

handleConnections :: Socket -> IO ()
handleConnections sock = do
  (handle, host, port) <- accept sock
  output <- hGetLine handle
  putStrLn output
  handleConnections sock

The main function starts a server on port 3001 and transfers the socket handler to a handleConnections function. It accepts any connection requests, reads the data, prints it to the server log, and waits for more clients.

Firstly, you need to compile the tcp-server.hs and tcp-client.hs files using GHC:

$ ghc --make tcp-server.hs
[1 of 1] Compiling Main             ( tcp-server.hs, tcp-server.o )
Linking tcp-server ...

$ ghc --make tcp-client.hs 
[1 of 1] Compiling Main             ( tcp-client.hs, tcp-client.o )
Linking tcp-client ...

You can now start the TCP server in a terminal:

$ ./tcp-server 
Starting server ...

You can then run the TCP client in another terminal:

$ ./tcp-client

You will now observe the “Hello, world” message printed in the terminal where the server is running:

$ ./tcp-server 
Starting server ...
Hello, world!

The Network.Socket package exposes more low-level socket functionality for Haskell and can be used if you need finer access and control. For example, consider the following UDP (User Datagram Protocol) client code:

-- udp-client.hs

import Network.Socket

main :: IO ()
main = withSocketsDo $ do
         (server:_) <- getAddrInfo Nothing (Just "localhost") (Just "3000")
         s <- socket (addrFamily server) Datagram defaultProtocol
         connect s (addrAddress server)
         send s "Hello, world!"
         sClose s

The getAddrInfo function resolves a host or service name to a network address. A UDP client connection is then requested for the server address, a message is sent, and the connection is closed. The type signatures of getAddrInfo, addrFamily, and addrAddress are given below:

ghci> :t getAddrInfo
getAddrInfo
  :: Maybe AddrInfo
     -> Maybe HostName -> Maybe ServiceName -> IO [AddrInfo]

ghci> :t addrFamily
addrFamily :: AddrInfo -> Family

ghci> :t addrAddress
addrAddress :: AddrInfo -> SockAddr

The corresponding UDP server code is as follows:

-- udp-server.hs

import Network.Socket

main :: IO ()
main = withSocketsDo $ do
         (server:_) <- getAddrInfo Nothing (Just "localhost") (Just "3000")
         s <- socket (addrFamily server) Datagram defaultProtocol
         bindSocket s (addrAddress server) >> return s
         putStrLn "Server started ..."
         handleConnections s

handleConnections :: Socket -> IO ()
handleConnections conn = do
  (text, _, _) <- recvFrom conn 1024
  putStrLn text
  handleConnections conn

The UDP server binds to localhost and starts to listen on port 3000. When a client connects, it reads a maximum of 1024 bytes of data, prints it to stdout, and waits to accept more connections. You can compile the udp-server.hs and udp-client.hs files using the following commands:

$ ghc --make udp-server.hs 
[1 of 1] Compiling Main             ( udp-server.hs, udp-server.o )
Linking udp-server ...

$ ghc --make udp-client.hs 
[1 of 1] Compiling Main             ( udp-client.hs, udp-client.o )
Linking udp-client ...

You can start the UDP server in one terminal:

$ ./udp-server 
Server started ...

You can then run the UDP client in another terminal:

$ ./tcp-client

You will now see the “Hello, world!” message printed in the terminal where the server is running:

$ ./udp-server 
Server started ...
Hello, world!

The network-uri module has many useful URI (Uniform Resource Identifier) parsing and test functions. You can install the same on Fedora using the following command:

$ cabal install network-uri

The parseURI function takes a string and attempts to convert it into a URI. It returns ‘Nothing’ if the input is not a valid URI, and returns the URI, otherwise. For example:

ghci> :m + Network.URI

ghci> parseURI "http://www.shakthimaan.com"
Just http://www.shakthimaan.com

ghci> parseURI "shakthimaan.com"
Nothing

The type signature of the parseURI function is given below:

ghci> :t parseURI
parseURI :: String -> Maybe URI

A number of functions are available for testing the input URI as illustrated in the following examples:

ghci> isURI "shakthimaan.com"
False

ghci> isURI "http://www.shakthimaan.com"
True

ghci> isRelativeReference "http://shakthimaan.com"
False

ghci> isRelativeReference "../about.html"
True

ghci> isAbsoluteURI "http://www.shakthimaan.com"
True

ghci> isAbsoluteURI "shakthimaan.com"
False

ghci> isIPv4address "192.168.100.2"
True

ghci> isIPv6address "2001:0db8:0a0b:12f0:0000:0000:0000:0001"
True

ghci> isIPv6address "192.168.100.2"
False

ghci> isIPv4address "2001:0db8:0a0b:12f0:0000:0000:0000:0001"
False

The type signatures of the above functions are as follows:

ghci> :t isURI
isURI :: String -> Bool

ghci> :t isRelativeReference
isRelativeReference :: String -> Bool

ghci> :t isAbsoluteURI
isAbsoluteURI :: String -> Bool

ghci> :t isIPv4address
isIPv4address :: String -> Bool

ghci> :t isIPv6address
isIPv6address :: String -> Bool

You can make a GET request for a URL and retrieve its contents. For example:

import Network
import System.IO

main = withSocketsDo $ do
    h <- connectTo "www.shakthimaan.com" (PortNumber 80)
    hSetBuffering h LineBuffering
    hPutStr h "GET / HTTP/1.1\nhost: www.shakthimaan.com\n\n"
    contents <- hGetContents h
    putStrLn contents
    hClose h

You can now compile and execute the above code, and it returns the index.html contents as shown below:

$ ghc --make get-network-uri.hs
[1 of 1] Compiling Main             ( get-network-uri.hs, get-network-uri.o )
Linking get-network-uri ...

$ ./get-network-uri 
HTTP/1.1 200 OK
Date: Sun, 05 Apr 2015 01:37:19 GMT
Server: Apache
Last-Modified: Tue, 08 Jul 2014 04:01:16 GMT
Accept-Ranges: bytes
Content-Length: 4604
Content-Type: text/html
...

You can refer to the network-uri package documentation at https://hackage.haskell.org/package/network-uri-2.6.0.1/docs/Network-URI.html for more detailed information.

The whois Haskell package allows you to query for information about hosting servers and domain names. You can install the package on Ubuntu, for example, using:

$ cabal install whois

The serverFor function returns a whois server that can be queried for obtaining more information regarding an IP or domain name. For example:

ghci> :m + Network.Whois

ghci> serverFor "shakthimaan.com"
Loading package array-0.4.0.1 ... linking ... done.
Loading package deepseq-1.3.0.1 ... linking ... done.
Loading package bytestring-0.10.0.2 ... linking ... done.
Loading package old-locale-1.0.0.5 ... linking ... done.
Loading package time-1.4.0.1 ... linking ... done.
Loading package unix-2.6.0.1 ... linking ... done.
Loading package network-2.6.0.2 ... linking ... done.
Loading package transformers-0.4.3.0 ... linking ... done.
Loading package mtl-2.2.1 ... linking ... done.
Loading package text-1.2.0.4 ... linking ... done.
Loading package parsec-3.1.9 ... linking ... done.
Loading package network-uri-2.6.0.1 ... linking ... done.
Loading package split-0.2.2 ... linking ... done.
Loading package whois-1.2.2 ... linking ... done.

Just (WhoisServer {hostname = "com.whois-servers.net", port = 43, query = "domain "})

You can use the above specific information with the whois1 function to make a DNS (Domain Name System) query:

ghci> whois1 "shakthimaan.com" WhoisServer {hostname = "com.whois-servers.net", port = 43, query = "domain "}
Just "\nWhois Server Version 2.0\n\nDomain names in the .com and .net domains can now be registered\n
...

You can also use the whois function to return information on the server as shown below:

ghci> whois "shakthimaan.com"
Just "\nWhois Server Version 2.0\n\nDomain names in the .com and .net domains can now be registered\n
...

The type signatures of serverFor, whois1 and whois functions are as follows:

ghc> :t serverFor
serverFor :: String -> Maybe WhoisServer

ghci> :t whois1
whois1 :: String -> WhoisServer -> IO (Maybe String)

ghci> :t whois
whois :: String -> IO (Maybe String, Maybe String)

The dns package provides a number of useful functions to make Domain Name System queries, and handle the responses. You can install the same on Ubuntu, for example, using the following commands:

$ sudo apt-get install zlib1g-dev
$ cabal install dns

A simple example of finding the IP addresses for the haskell.org domain is shown below:

ghci> import Network.DNS.Lookup
ghci> import Network.DNS.Resolver

ghci> let hostname = Data.ByteString.Char8.pack "www.haskell.org"

ghci> rs <- makeResolvSeed defaultResolvConf

ghci> withResolver rs $ \resolver -> lookupA resolver hostname
Right [108.162.203.60,108.162.204.60]

The defaultResolvConf is of type ResolvConf and consists of the following default values:

--     * 'resolvInfo' is 'RCFilePath' \"\/etc\/resolv.conf\".
--
--     * 'resolvTimeout' is 3,000,000 micro seconds.
--
--     * 'resolvRetry' is 3.

The makeResolvSeed, and withResolver functions assist in making the actual DNS resolution. The lookupA function obtains all the A records for the DNS entry. Their type signatures are shown below:

ghci> :t makeResolvSeed
makeResolvSeed :: ResolvConf -> IO ResolvSeed

ghci> :t withResolver
withResolver :: ResolvSeed -> (Resolver -> IO a) -> IO a

ghci> :t lookupA
lookupA
  :: Resolver
     -> dns-1.4.5:Network.DNS.Internal.Domain
     -> IO
          (Either
             dns-1.4.5:Network.DNS.Internal.DNSError
             [iproute-1.4.0:Data.IP.Addr.IPv4])

The lookupAAAA function returns all the IPv6 ‘AAAA’ records for the domain. For example:

ghci> withResolver rs $ \resolver -> lookupAAAA resolver hostname
Right [2400:cb00:2048:1::6ca2:cc3c,2400:cb00:2048:1::6ca2:cb3c]

Its type signature is shown below:

lookupAAAA
  :: Resolver
     -> dns-1.4.5:Network.DNS.Internal.Domain
     -> IO
          (Either
             dns-1.4.5:Network.DNS.Internal.DNSError
             [iproute-1.4.0:Data.IP.Addr.IPv6])

The MX records for the hostname can be returned using the lookupMX function. An example for the shakthimaan.com website is as follows:

ghci> import Network.DNS.Lookup
ghci> import Network.DNS.Resolver

ghci> let hostname = Data.ByteString.Char8.pack "www.shakthimaan.com"

ghci> rs <- makeResolvSeed defaultResolvConf

ghci> withResolver rs $ \resolver -> lookupMX resolver hostname
Right [("shakthimaan.com.",0)]

The type signature of the lookupMX function is as under:

ghci> :t lookupMX
lookupMX
  :: Resolver
     -> dns-1.4.5:Network.DNS.Internal.Domain
     -> IO
          (Either
             dns-1.4.5:Network.DNS.Internal.DNSError
             [(dns-1.4.5:Network.DNS.Internal.Domain, Int)])

The nameservers for the domain can be returned using the lookupNS function. For example:

ghci> withResolver rs $ \resolver -> lookupNS resolver hostname
Right ["ns22.webhostfreaks.com.","ns21.webhostfreaks.com."]

The type signature of the lookupNS function is shown below:

ghci> :t lookupNS
lookupNS
  :: Resolver
     -> dns-1.4.5:Network.DNS.Internal.Domain
     -> IO
          (Either
             dns-1.4.5:Network.DNS.Internal.DNSError
             [dns-1.4.5:Network.DNS.Internal.Domain])

You can also return the entire DNS response using the lookupRaw function as illustrated below:

ghci> :m + Network.DNS.Types

ghci> let hostname = Data.ByteString.Char8.pack "www.ubuntu.com"

ghci> rs <- makeResolvSeed defaultResolvConf

ghci> withResolver rs $ \resolver -> lookupRaw resolver hostname A
Right (DNSFormat 
  {header = DNSHeader 
    {identifier = 29504, 
     flags = DNSFlags 
       {qOrR = QR_Response, 
        opcode = OP_STD, 
        authAnswer = False, 
        trunCation = False, 
        recDesired = True, 
        recAvailable = True, 
        rcode = NoErr}, 
     qdCount = 1, 
     anCount = 1, 
     nsCount = 3, 
     arCount = 3}, 
   question = [
     Question 
       {qname = "www.ubuntu.com.", 
        qtype = A}], 
     answer = [
       ResourceRecord 
         {rrname = "www.ubuntu.com.", 
          rrtype = A, 
          rrttl = 61, 
          rdlen = 4, 
          rdata = 91.189.89.103}], 
     authority = [
       ResourceRecord 
         {rrname = "ubuntu.com.", 
          rrtype = NS, 
          rrttl = 141593, 
          rdlen = 16, 
          rdata = ns2.canonical.com.},
       ResourceRecord 
         {rrname = "ubuntu.com.", 
          rrtype = NS, 
          rrttl = 141593, 
          rdlen = 6, 
          rdata = ns1.canonical.com.},
       ResourceRecord
         {rrname = "ubuntu.com.", 
          rrtype = NS, 
          rrttl = 141593, 
          rdlen = 6, 
          rdata = ns3.canonical.com.}], 
    additional = [
      ResourceRecord 
        {rrname = "ns2.canonical.com.", 
         rrtype = A, 
         rrttl = 88683, 
         rdlen = 4, 
         rdata = 91.189.95.3},
      ResourceRecord 
        {rrname = "ns3.canonical.com.", 
         rrtype = A, 
         rrttl = 88683, 
         rdlen = 4, 
         rdata = 91.189.91.139},
      ResourceRecord 
        {rrname = "ns1.canonical.com.", 
         rrtype = A, 
         rrttl = 88683, 
         rdlen = 4, 
         rdata = 91.189.94.173}]})

Please refer the Network.DNS hackage web page https://hackage.haskell.org/package/dns for more information.

[Published in Open Source For You (OSFY) magazine, April 2015 edition.]

In this article we shall explore access to Redis and PostgreSQL databases using Haskell modules. The hackage website at https://hackage.haskell.org/packages/#cat:Database provides a vast number of database packages that you can use, a couple of which will be covered here..

You will need to install the cabal-install tool on Fedora, for example, using the following command:

$ sudo yum install cabal-install
  • Connecting to the Redis database

Let’s use the hedis package to connect to the Redis server. Install the Fedora dependency package alex, and the Redis server as shown below:

$ sudo yum install alex redis

You can then install the hedis package using the following commands:

$ cabal update
$ cabal install hedis

This installs the latest hedis version 0.6.5. You can now start the Redis server on Fedora using the service command:

$ sudo service redis start

You can then test connectivity to the Redis server using the redis-cli command by issuing the PING command as follows:

$ redis-cli

127.0.0.1:6379> PING
PONG

You can also test the same using the hedis package inside the GHCi interpreter as illustrated below:

$ ghci
GHCi, version 7.6.3: http://www.haskell.org/ghc/  :? for help
Loading package ghc-prim ... linking ... done.
Loading package integer-gmp ... linking ... done.
Loading package base ... linking ... done.

ghci> :m Database.Redis 

ghci> conn <- connect defaultConnectInfo

Loading package array-0.4.0.1 ... linking ... done.
Loading package base-unicode-symbols-0.2.2.4 ... linking ... done.
Loading package deepseq-1.3.0.1 ... linking ... done.
Loading package old-locale-1.0.0.5 ... linking ... done.
Loading package time-1.4.0.1 ... linking ... done.
Loading package transformers-0.3.0.0 ... linking ... done.
Loading package bytestring-0.10.0.2 ... linking ... done.
Loading package text-0.11.3.1 ... linking ... done.
Loading package stm-2.4.2 ... linking ... done.
Loading package primitive-0.5.0.1 ... linking ... done.
Loading package vector-0.10.0.1 ... linking ... done.
Loading package hashable-1.1.2.5 ... linking ... done.
Loading package transformers-base-0.4.1 ... linking ... done.
Loading package monad-control-0.3.2.1 ... linking ... done.
Loading package containers-0.5.0.0 ... linking ... done.
Loading package attoparsec-0.10.4.0 ... linking ... done.
Loading package mtl-2.1.2 ... linking ... done.
Loading package BoundedChan-1.0.3.0 ... linking ... done.
Loading package bytestring-lexing-0.4.3.2 ... linking ... done.
Loading package unix-2.6.0.1 ... linking ... done.
Loading package network-2.6.0.2 ... linking ... done.
Loading package resource-pool-0.2.3.2 ... linking ... done.
Loading package hedis-0.6.5 ... linking ... done.

ghci> runRedis conn ping
Right Pong

It is recommended that you use defaultConnectInfo to connect to the database, and its type is ConnectInfo:

ghci> :t defaultConnectInfo
defaultConnectInfo :: ConnectInfo

The different options that can be used in defaultConnectInfo are as follows:

connectHost           = "localhost"
connectPort           = PortNumber 6379 -- Redis port
connectAuth           = Nothing         -- No authentication
connectDatabase       = 0               -- SELECT database 0
connectMaxConnections = 10              -- Up to 10 connections
connectMaxIdleTime    = 20              -- Keep connection open for 20 seconds

The types of conn, connect, runRedis and ping are given below:

ghci> :t conn
conn :: Connection

ghci> :t connect
connect :: ConnectInfo -> IO Connection

ghci> :t runRedis
runRedis :: Connection -> Redis a -> IO a

ghci> :t ping
ping :: RedisCtx m f => m (f Status)

If the Redis server was not started, and you tried to issue the ping command, the following exception will be automatically thrown by the package:

ghci> runRedis conn ping
*** Exception: connect: does not exist (No route to host)

You can automate the above code snippets into Haskell code with a main function as demonstrated below:

{-# LANGUAGE OverloadedStrings #-}
import Database.Redis

main :: IO (Either Reply Status)
main = do
  conn <- connect defaultConnectInfo
  runRedis conn ping

The OverloadedStrings extension allows string literals to be polymorphic for the IsString class. You can compile and run the above code inside GHCi, as follows:

$ ghci ping.hs

GHCi, version 7.6.3: http://www.haskell.org/ghc/  :? for help
Loading package ghc-prim ... linking ... done.
Loading package integer-gmp ... linking ... done.
Loading package base ... linking ... done.
[1 of 1] Compiling Main             ( ping.hs, interpreted )
Ok, modules loaded: Main.

ghci> main
...
Right Pong

The echo Redis command is used to print a message that is passed as an argument to it. The equivalent hedis echo command expects the message to be of type ByteString. For example:

{-# LANGUAGE OverloadedStrings #-}
import Database.Redis
import qualified Data.ByteString as B

bytes :: B.ByteString
bytes = "Hello, World" :: B.ByteString

main :: IO (Either Reply B.ByteString)
main = do
  conn <- connect defaultConnectInfo
  runRedis conn $ echo bytes

Loading the above code in GHCi produces the following output:

ghci> main
Right "Hello, World"

The type signature of the echo function is as follows:

echo
  :: RedisCtx m f =>
     Data.ByteString.Internal.ByteString
     -> m (f Data.ByteString.Internal.ByteString)

You can set a value to a key using the set function in hedis. An example is shown below:

{-# LANGUAGE OverloadedStrings #-}
import Database.Redis

main :: IO (Either Reply Status)
main = do
  conn <- connect defaultConnectInfo
  runRedis conn $ set "a" "apple"

Loading the above set.hs code in GHCi and testing the same produces the following output:

ghci> :l set.hs
[1 of 1] Compiling Main             ( set.hs, interpreted )
Ok, modules loaded: Main.

ghci> main
Right Ok

The type signature of the set function is shown below:

ghci> :t set
set
  :: RedisCtx m f =>
     Data.ByteString.Internal.ByteString
     -> Data.ByteString.Internal.ByteString -> m (f Status)

You can verify the value of the key `a’ from the redis-cli command, and it must return the value “apple”:

127.0.0.1:6379> get a
"apple"

You can also retrieve the value of a key using the get function. For example:

{-# LANGUAGE OverloadedStrings #-}
import Database.Redis
import Control.Monad.IO.Class

main :: IO ()
main = do
  conn <- connect defaultConnectInfo
  runRedis conn $ do
         result <- get "a"
         liftIO $ print result

Executing the above code in GHCi gives the expected result:

ghci> :l get.hs
[1 of 1] Compiling Main             ( get.hs, interpreted )
Ok, modules loaded: Main.

ghci> main
Right (Just "apple")

The liftIO function transforms an IO action into a Monad. Its type signature is shown below:

ghci> :t liftIO
liftIO :: MonadIO m => IO a -> m a

The type signature of the get function is as follows:

ghci> :t get
get
  :: RedisCtx m f =>
     Data.ByteString.Internal.ByteString
     -> m (f (Maybe Data.ByteString.Internal.ByteString))

You are encouraged to read the Database.Redis documentation page that contains a comprehensive list of commands and their usage at https://hackage.haskell.org/package/hedis-0.6.5/docs/Database-Redis.html.

  • Accessing the PostgreSQL database

We shall now explore accessing a PostgreSQL database using the postgresql-simple (0.4.10.0) package. You will need to install and configure PostgreSQL for your GNU/Linux distribution. Please follow your distribution documentation to do so. On Fedora, for example, you can install the database server using the following command:

$ sudo yum install postgresql-server postgresql-contrib

You can then start the database server using the following service command:

$ sudo service postgresql start

You can now install the postgresql-simple package using the cabal command:

$ cabal install postgresql-simple

Let us first create a database and a schema using the Postgresql command-line utility psql:

$ psql -U postgres
Password for user postgres: 
psql (9.3.5)
Type "help" for help.

postgres=# \l
                                  List of databases
   Name    |  Owner   | Encoding |   Collate   |    Ctype    |   Access privileges   
-----------+----------+----------+-------------+-------------+-----------------------
 postgres  | postgres | UTF8     | en_US.UTF-8 | en_US.UTF-8 | 
 template0 | postgres | UTF8     | en_US.UTF-8 | en_US.UTF-8 | =c/postgres          +
           |          |          |             |             | postgres=CTc/postgres
 template1 | postgres | UTF8     | en_US.UTF-8 | en_US.UTF-8 | =c/postgres          +
           |          |          |             |             | postgres=CTc/postgres
(3 rows)

postgres=# CREATE DATABASE test;
CREATE DATABASE

postgres-# \c test
You are now connected to database "test" as user "postgres".

test=# create schema social;
CREATE SCHEMA

test=# \dn
 public | postgres
 social | postgres

We can then create a users’ table with an id, first name and last name using the postgresql-simple package.

{-# LANGUAGE OverloadedStrings #-}

import Database.PostgreSQL.Simple

main :: IO ()
main = do
  conn <- connect defaultConnectInfo
    { connectUser = "postgres"
    , connectPassword = "postgres123"
    , connectDatabase = "test"
    }

  execute conn "create table social.users (id INT, fname VARCHAR(80), lname VARCHAR(80))" ()

  close conn

Loading the above code in GHCi creates the table social.users as shown below:

$ ghci create.hs 

GHCi, version 7.6.3: http://www.haskell.org/ghc/  :? for help
Loading package ghc-prim ... linking ... done.
Loading package integer-gmp ... linking ... done.
Loading package base ... linking ... done.
[1 of 1] Compiling Main             ( create.hs, interpreted )
Ok, modules loaded: Main.

ghci> main

Loading package array-0.4.0.1 ... linking ... done.
Loading package deepseq-1.3.0.1 ... linking ... done.
Loading package bytestring-0.10.0.2 ... linking ... done.
Loading package containers-0.5.0.0 ... linking ... done.
Loading package text-0.11.3.1 ... linking ... done.
Loading package attoparsec-0.10.4.0 ... linking ... done.
Loading package blaze-builder-0.3.1.1 ... linking ... done.
Loading package dlist-0.5 ... linking ... done.
Loading package hashable-1.1.2.5 ... linking ... done.
Loading package transformers-0.3.0.0 ... linking ... done.
Loading package mtl-2.1.2 ... linking ... done.
Loading package old-locale-1.0.0.5 ... linking ... done.
Loading package syb-0.4.0 ... linking ... done.
Loading package pretty-1.1.1.0 ... linking ... done.
Loading package template-haskell ... linking ... done.
Loading package time-1.4.0.1 ... linking ... done.
Loading package unordered-containers-0.2.3.0 ... linking ... done.
Loading package primitive-0.5.0.1 ... linking ... done.
Loading package vector-0.10.0.1 ... linking ... done.
Loading package aeson-0.6.2.1 ... linking ... done.
Loading package random-1.0.1.1 ... linking ... done.
Loading package scientific-0.2.0.2 ... linking ... done.
Loading package case-insensitive-1.0.0.1 ... linking ... done.
Loading package blaze-textual-0.2.0.8 ... linking ... done.
Loading package postgresql-libpq-0.9.0.2 ... linking ... done.
Loading package binary-0.7.4.0 ... linking ... done.
Loading package cereal-0.3.5.2 ... linking ... done.
Loading package entropy-0.2.2.1 ... linking ... done.
Loading package tagged-0.6 ... linking ... done.
Loading package crypto-api-0.11 ... linking ... done.
Loading package cryptohash-0.9.0 ... linking ... done.
Loading package network-info-0.2.0.5 ... linking ... done.
Loading package uuid-1.3.8 ... linking ... done.
Loading package postgresql-simple-0.4.10.0 ... linking ... done.

You can verify the created table from the psql prompt:

test=# \d social.users
 id     | integer               | 
 fname  | character varying(80) | 
 lname  | character varying(80) | 

You can also list the databases in the PostgreSQL server using the query_ function as illustrated below:

{-# LANGUAGE OverloadedStrings #-}

import Database.PostgreSQL.Simple

main :: IO ()
main = do
  conn <- connect defaultConnectInfo
    { connectUser = "postgres"
    , connectPassword = "postgres123"
    , connectDatabase = "test"
    }
  databases <- query_ conn "SELECT datname FROM pg_database"
  print (databases :: [Only String])

  close conn

Executing the above code in GHCi produces the following output:

$ ghci show.hs 
GHCi, version 7.6.3: http://www.haskell.org/ghc/  :? for help
Loading package ghc-prim ... linking ... done.
Loading package integer-gmp ... linking ... done.
Loading package base ... linking ... done.
[1 of 1] Compiling Main             ( show.hs, interpreted )
Ok, modules loaded: Main.

ghci> main
[Only {fromOnly = "template1"},Only {fromOnly = "template0"},Only {fromOnly = "postgres"},Only {fromOnly = "test"}]

You can now insert a record into the databaes using the execute function:

execute conn "insert into social.users (id, fname, lname) values (?, ?, ?)" ["1" :: String, "Edwin" :: String, "Brady" :: String]

After executing the above code, you can verify the database entry from the psql prompt:

test=# select * from social.users;
 id | fname | lname 
----+-------+-------
  1 | Edwin | Brady
(1 row)

You can also do batch inserts using the executeMany function. For example:

executeMany conn "insert into social.users (id, fname, lname) values (?, ?, ?)" [("2" :: String, "Simon" :: String, "Marlow" :: String), ("3" :: String, "Ulf" :: String, "Norell" :: String)]

After running the above code, you can check the newly added rows in the database from the psql command-line tool:

test=# select * from social.users;
 id | fname | lname  
----+-------+--------
  1 | Edwin | Brady
  2 | Simon | Marlow
  3 | Ulf   | Norell
(3 rows)

You can also change a record entry using the UPDATE statement as shown below:

execute conn "update social.users SET lname = 'Peyton Jones' where fname = 'Simon'" ()

The corresponding entry is updated as seen from the psql prompt:

test=# select * from social.users;
 id | fname |    lname     
----+-------+--------------
  1 | Edwin | Brady
  3 | Ulf   | Norell
  2 | Simon | Peyton Jones
(3 rows)

It is recommended to catch exceptions when running database commands. Consider the following example, where the number of arguments passed does not match with the expected:

{-# LANGUAGE OverloadedStrings #-}

import Database.PostgreSQL.Simple
import Control.Exception
import GHC.Int

main :: IO ()
main = do
  conn <- connect defaultConnectInfo
    { connectUser = "postgres"
    , connectPassword = "postgres123"
    , connectDatabase = "test"
    }
  result <- try (execute conn "insert into social.users (id, fname, lname) values (?, ?, ?)" ["4" :: String, "Laurel" :: String]) :: IO (Either SomeException Int64)
  case result of
      Left ex  -> putStrLn $ "Caught exception: " ++ show ex
      Right val -> putStrLn $ "The answer was: " ++ show val
  close conn

The error is observed when the main function is executed as shown below:

ghci> main
Caught exception: FormatError {fmtMessage = "3 '?' characters, but 2 parameters", fmtQuery = "insert into social.users (id, fname, lname) values (?, ?, ?)", fmtParams = ["4","Laurel"]}

You can also retrieve multiple records from the database and use the results using a map function. An example is illustrated below:

{-# LANGUAGE OverloadedStrings #-}

import Database.PostgreSQL.Simple
import Control.Monad
import Data.Text as Text

main :: IO ()
main = do
  conn <- connect defaultConnectInfo
    { connectUser = "postgres"
    , connectPassword = "postgres123"
    , connectDatabase = "test"
    }
  users <- query_ conn "SELECT fname, lname FROM social.users"
  forM_ users $ \(fname, lname) ->
      putStrLn $ Text.unpack fname ++ " " ++ Text.unpack lname
  close conn

The output after executing the above code in GHCi returns the actual data:

ghci> main
Edwin Brady
Ulf Norell
Simon Peyton Jones

Please refer to the Database.PostgreSQL.Simple documentation for more examples and usage at https://hackage.haskell.org/package/postgresql-simple-0.4.10.0/docs/Database-PostgreSQL-Simple.html.

[Published in Open Source For You (OSFY) magazine, March 2015 edition.]

In this ninth article on Haskell, I shall cover access to Sqlite, and MySQL databases using Haskell modules. A number of packages are available from the https://hackage.haskell.org/packages/#cat:Database website, but, I will illustrate a few of them with examples.

You first need to install the cabal-install tool on Fedora, for example, using the following command:

$ sudo yum install cabal-install

You can then install HDBC.Sqlite3 using the following command:

$ cabal install HDBC-sqlite3

This installs the latest 2.3.3.0 version from https://hackage.haskell.org/package/HDBC-sqlite3. You can also install the Sqlite3 package on Fedora for testing, as follows:

$ sudo yum install sqlite

To initiate a connection to a database, you can test it out in the GHCi prompt using the connectSqlite3 function, as shown below:

$ ghci

GHCi, version 7.6.3: http://www.haskell.org/ghc/  :? for help
Loading package ghc-prim ... linking ... done.
Loading package integer-gmp ... linking ... done.
Loading package base ... linking ... done.

ghci> :module Database.HDBC Database.HDBC.Sqlite3

ghci> conn <- connectSqlite3 "students.db"

Loading package array-0.4.0.1 ... linking ... done.
Loading package deepseq-1.3.0.1 ... linking ... done.
Loading package old-locale-1.0.0.5 ... linking ... done.
Loading package time-1.4.0.1 ... linking ... done.
Loading package transformers-0.3.0.0 ... linking ... done.
Loading package bytestring-0.10.0.2 ... linking ... done.
Loading package text-0.11.3.1 ... linking ... done.
Loading package old-time-1.1.0.1 ... linking ... done.
Loading package containers-0.5.0.0 ... linking ... done.
Loading package utf8-string-0.3.7 ... linking ... done.
Loading package mtl-2.1.2 ... linking ... done.
Loading package convertible-1.1.0.0 ... linking ... done.
Loading package HDBC-2.4.0.0 ... linking ... done.
Loading package HDBC-sqlite3-2.3.3.0 ... linking ... done.

The signature of the connectSqlite3 function is as follows:

ghci> :t connectSqlite3
connectSqlite3 :: FilePath -> IO Connection

The type of conn is a Connection.

ghci> :t conn
conn :: Connection

If you already have an existing Sqlite3 database, you can give the full path to the database and connect to it, or else you can now create a table using the Sqlite CREATE TABLE syntax as shown below:

ghci> run conn "CREATE TABLE names (id INTEGER NOT NULL, fname VARCHAR(80), lname VARCHAR(80))" []
0

The type signature of run is as follows:

ghci> :t run
run
  :: IConnection conn => conn -> String -> [SqlValue] -> IO Integer

It takes three arguments as input and performs an IO computation that returns an integer indicating the status of the execution. The first argument to run is the connection, the second argument is the Sqlite command to be executed, and finally is the array of SqlValues that provide a mapping between Haskell values and SQL databases.

Both Haskell and SQL databases have types, and different databases may have different representations of the types. In order to provide a consistent mapping between the two, each HDBC driver implements the relation using SqlValue.

You can now insert a record into the database using the following command:

ghci> run conn "INSERT INTO names (id, fname, lname) VALUES(1, 'Edwin', 'Brady')" []
1

ghci> commit conn

The type signature of commit is given here:

ghci> :t commit
commit :: IConnection conn => conn -> IO ()

It takes a connection and completes the pending IO actions. To read the result from Haskell you can use the quickQuery function from the GHCi prompt, as follows:

ghci> quickQuery conn "SELECT * from names" []
[[SqlByteString "1",SqlByteString "Edwin",SqlByteString "Brady"]]

The type signature of the quickQuery function is as follows:

quickQuery
  :: IConnection conn =>
     conn -> String -> [SqlValue] -> IO [[SqlValue]]

You can also verify the result of the above actions using the sqlite3 executable in the command prompt as illustrated below:

$ sqlite3 students.db 

SQLite version 3.8.4.3 2014-04-03 16:53:12
Enter ".help" for usage hints.

sqlite> .schema
CREATE TABLE names (id INTEGER NOT NULL, fname VARCHAR(80), lname VARCHAR(80));

sqlite> select * from names;
1|Edwin|Brady

You can also do batch processing for inserts by preparing the statements and executing them:

ghci> batch <- prepare conn "INSERT INTO names VALUES (?, ?, ?)"

ghci> execute batch [toSql (2 :: Int), toSql "Simon", toSql "Marlow"]
1

ghci> execute batch [toSql (3 :: Int), toSql "Ulf", toSql "Norell"]
1

ghci> commit conn

The type signatures of the prepare and execute functions are given below:

ghci> :t prepare
prepare :: IConnection conn => conn -> String -> IO Statement

ghci> :t execute
execute :: Statement -> [SqlValue] -> IO Integer

You can once again check the records in the database using the quickQuery function:

ghci> quickQuery' conn "SELECT * from names" []

[[SqlByteString "1",SqlByteString "Edwin",SqlByteString "Brady"],[SqlByteString "2",SqlByteString "Simon",SqlByteString "Marlow"],[SqlByteString "3",SqlByteString "Ulf",SqlByteString "Norell"]]

You can also run an update query to the database. For example:

ghci> run conn "UPDATE names set lname = 'Peyton Jones' WHERE fname = 'Simon'" []
1

ghci> commit conn

Verifying the output from the Sqlite3 command prompt, you get:

sqlite> select * from names;

1|Edwin|Brady
2|Simon|Peyton Jones
3|Ulf|Norell

The HDBC driver provides many functions to retrieve information regarding the database and the drivers. A few examples are illustrated below:

ghci> hdbcDriverName conn
"sqlite3"

ghci> hdbcClientVer conn
"3.8.4.3"

ghci> dbTransactionSupport conn
True

ghci> getTables conn
["names"]

ghci> describeTable conn "names"

[("id",SqlColDesc {colType = SqlIntegerT, colSize = Nothing, colOctetLength = Nothing, colDecDigits = Nothing, colNullable = Nothing}),("fname",SqlColDesc {colType = SqlUnknownT "varchar(80)", colSize = Nothing, colOctetLength = Nothing, colDecDigits = Nothing, colNullable = Nothing}),("lname",SqlColDesc {colType = SqlUnknownT "varchar(80)", colSize = Nothing, colOctetLength = Nothing, colDecDigits = Nothing, colNullable = Nothing})]

It is considered good practice to use handleSqlError before running any HDBC commands to catch errors that may arise during the database transactions. For example:

ghci> handleSqlError $ quickQuery conn "SELECT * from namesaaa" []

*** Exception: user error (SQL error: SqlError {seState = "", seNativeError = 1, seErrorMsg = "prepare 23: SELECT * from namesaaa: no such table: namesaaa"})

To disconnect from the database, you can use the disconnect function provided by HDBC as shown below:

ghci> disconnect conn

Let’s now look at how to access a MySQL database using the mysql-simple package. You will also need to create or grant privileges for an existing user to use the MySQL database server. Please follow your GNU/Linux distribution manual on how to install and configure a MySQL server. On Fedora, for example, you must have mysql and mysql-server installed:

$ sudo yum install community-mysql community-mysql-server

You can install the mysql-simple Haskell package using:

$ cabal install mysql-simple

Create a test database using the mysql command line tool as shown below:

$ mysql -u user -p

Enter password: 
Welcome to the MySQL monitor.  Commands end with ; or \g.
Your MySQL connection id is 3
Server version: 5.5.38-log MySQL Community Server (GPL)

Copyright (c) 2000, 2014, Oracle and/or its affiliates. All rights reserved.

Oracle is a registered trademark of Oracle Corporation and/or its
affiliates. Other names may be trademarks of their respective
owners.

Type 'help;' or '\h' for help. Type '\c' to clear the current input statement.

mysql> create database test;
Query OK, 1 row affected (0.03 sec)

Let us create a users’ table that has an id as well as first name and last name fields, using the mysql-simple package, as follows:

{-# LANGUAGE OverloadedStrings #-}

import Database.MySQL.Simple

main :: IO ()
main = do
  conn <- connect defaultConnectInfo
    { connectUser = "user"
    , connectPassword = "password"
    , connectDatabase = "test"
    }

  execute conn "create table users (id INT, fname VARCHAR(80), lname VARCHAR(80))" ()

  close conn

The OverloadedStrings extension allows string literals to be polymorphic for the IsString class. The defaultConnectInfo is of type ConnectInfo:

ghci> :t defaultConnectInfo
defaultConnectInfo :: ConnectInfo

ConnectInfo can take many parameters to describe the connectivity to the MySQL server. For example:

connectInfo :: ConnectInfo
connectInfo = ConnectInfo { connectHost = "localhost",
                            connectPort = 3306,
                            connectUser = "user",
                        connectPassword = "password",
                        connectDatabase = "test",
                         connectOptions = [],
                            connectPath = "",
                             connectSSL = Nothing }

The above code to create a table can be compiled directly in GHCi and the main function can be executed as given below:

$ ghci create.hs

GHCi, version 7.6.3: http://www.haskell.org/ghc/  :? for help
Loading package ghc-prim ... linking ... done.
Loading package integer-gmp ... linking ... done.
Loading package base ... linking ... done.
[1 of 1] Compiling Main             ( create.hs, interpreted )
Ok, modules loaded: Main.

ghci> main

Loading package array-0.4.0.1 ... linking ... done.
Loading package deepseq-1.3.0.1 ... linking ... done.
Loading package old-locale-1.0.0.5 ... linking ... done.
Loading package time-1.4.0.1 ... linking ... done.
Loading package bytestring-0.10.0.2 ... linking ... done.
Loading package text-0.11.3.1 ... linking ... done.
Loading package blaze-builder-0.3.1.1 ... linking ... done.
Loading package primitive-0.5.0.1 ... linking ... done.
Loading package vector-0.10.0.1 ... linking ... done.
Loading package pcre-light-0.4 ... linking ... done.
Loading package containers-0.5.0.0 ... linking ... done.
Loading package attoparsec-0.10.4.0 ... linking ... done.
Loading package blaze-textual-0.2.0.8 ... linking ... done.
Loading package base16-bytestring-0.1.1.6 ... linking ... done.
Loading package mysql-0.1.1.7 ... linking ... done.
Loading package mysql-simple-0.2.2.4 ... linking ... done.

You can check with the mysql command line utility for the created table:

mysql> use test;
Reading table information for completion of table and column names
You can turn off this feature to get a quicker startup with -A

Database changed
mysql> desc users;
+-------+-------------+------+-----+---------+-------+
| Field | Type        | Null | Key | Default | Extra |
+-------+-------------+------+-----+---------+-------+
| id    | int(11)     | YES  |     | NULL    |       |
| fname | varchar(80) | YES  |     | NULL    |       |
| lname | varchar(80) | YES  |     | NULL    |       |
+-------+-------------+------+-----+---------+-------+
3 rows in set (0.00 sec)

You can now list the databases available in the MySQL server using the query_ function as illustrated below:

{-# LANGUAGE OverloadedStrings #-}

import Database.MySQL.Simple

main :: IO ()
main = do
  conn <- connect defaultConnectInfo
    { connectUser = "user"
    , connectPassword = "password"
    , connectDatabase = "test"
    }
  databases <- query_ conn "SHOW databases"
  print (databases :: [Only String])

  close conn

You can compile the above code directly with GHCi and execute the main function, as follows:

 $ ghci show-databases.hs

GHCi, version 7.6.3: http://www.haskell.org/ghc/  :? for help
Loading package ghc-prim ... linking ... done.
Loading package integer-gmp ... linking ... done.
Loading package base ... linking ... done.
[1 of 1] Compiling Main             ( show-databases.hs, interpreted )
Ok, modules loaded: Main.

ghci> main

Loading package array-0.4.0.1 ... linking ... done.
Loading package deepseq-1.3.0.1 ... linking ... done.
...
[Only {fromOnly = "information_schema"},Only {fromOnly = "mysql"},Only {fromOnly = "performance_schema"},Only {fromOnly = "test"}]

You can try inserting a record into the database using the execute function:

execute conn "insert into users (id, fname, lname) values (?, ?, ?)" ["1" :: String, "Edwin" :: String, "Brady" :: String]

After running the code, you can check the database entry using the mysql client program as shown below:

mysql> select * from users;

+------+-------+-------+
| id   | fname | lname |
+------+-------+-------+
|    1 | Edwin | Brady |
+------+-------+-------+
1 row in set (0.00 sec)

You can also do batch inserts using the executeMany function. For example:

executeMany conn "insert into users (id, fname, lname) values (?, ?, ?)" [("2" :: String, "Simon" :: String, "Marlow" :: String), ("3" :: String, "Ulf" :: String, "Norell" :: String)]

You can verify the execution of the code from the mysql utility:

mysql> select * from users;
+------+-------+--------+
| id   | fname | lname  |
+------+-------+--------+
|    1 | Edwin | Brady  |
|    2 | Simon | Marlow |
|    3 | Ulf   | Norell |
+------+-------+--------+
3 rows in set (0.01 sec)

You can change a record entry using the UPDATE MySQL command:

execute conn "update users SET lname = 'Peyton Jones' where fname = 'Simon'" ()

Executing the code in GHCi, and checking the results with the mysql prompt gives the following changed output:

mysql> select * from users;
+------+-------+--------------+
| id   | fname | lname        |
+------+-------+--------------+
|    1 | Edwin | Brady        |
|    2 | Simon | Peyton Jones |
|    3 | Ulf   | Norell       |
+------+-------+--------------+
3 rows in set (0.00 sec)

It is important to catch any exceptions that may arise on executing the database commands. Consider the following example, where the number of arguments passed does not match with the expected:

{-# LANGUAGE OverloadedStrings #-}

import Database.MySQL.Simple
import Control.Exception
import GHC.Int

main :: IO ()
main = do
  conn <- connect defaultConnectInfo
    { connectUser = "user"
    , connectPassword = "password"
    , connectDatabase = "test"
    }
  result <- try (execute conn "insert into users (id, fname, lname) values (?, ?, ?)" ["4" :: String, "Laurel" :: String]) :: IO (Either SomeException Int64)
  case result of
      Left ex  -> putStrLn $ "Caught exception: " ++ show ex
      Right val -> putStrLn $ "The answer was: " ++ show val
  close conn

The error is caught when the main function is executed inside GHCi:

ghci> main
Caught exception: FormatError {fmtMessage = "3 '?' characters, but 2 parameters", fmtQuery = "insert into users (id, fname, lname) values (?, ?, ?)", fmtParams = ["4","Laurel"]}

You can also map through the results returned from the database and use them for your needs. The following is an illustration of the same:

{-# LANGUAGE OverloadedStrings #-}

import Database.MySQL.Simple
import Control.Monad
import Data.Text as Text

main :: IO ()
main = do
  conn <- connect defaultConnectInfo
    { connectUser = "user"
    , connectPassword = "password"
    , connectDatabase = "test"
    }
  users <- query_ conn "SELECT fname, lname FROM users"
  forM_ users $ \(fname, lname) ->
      putStrLn $ Text.unpack fname ++ " " ++ Text.unpack lname
  close conn

The resultant output when executing the main function in GHCi is given below:

ghci> main

Edwin Brady
Simon Peyton Jones
Ulf Norell

You are encouraged to read Database.MySQL.Simple documentation from https://hackage.haskell.org/package/mysql-simple-0.2.2.4/docs/Database-MySQL-Simple.html for more information.

[Published in Open Source For You (OSFY) magazine, February 2015 edition.]

Let’s take a look at the property-based testing of Haskell programs and at the Cabal tool, which is used to build and manage Haskell packages and applications.

One of the main features of testing in Haskell is property-based testing. The type system allows you to infer and derive types, and also helps in auto-generating test cases. QuickCheck is a popular property-based testing library for Haskell. If your program is pure, you can write tests to ascertain the properties and invariants of your programs, and the tests can be auto-generated and executed.

You can install QuickCheck on Fedora, for example, by using the following command:

$ sudo yum install ghc-QuickCheck-devel

Consider a simple function to add two integers:

mySum :: Int -> Int -> Int
mySum a b = a + b

We can ascertain the property of the function that ‘a + b’ is the same as ‘b + a’ using the QuickCheck library. You must first define the invariant in a function as shown below:

prop_mySum a b = mySum a b == mySum b a

You can test the code directly in the GHCi prompt, using the following command:

$ ghci sum.hs 
GHCi, version 7.6.3: http://www.haskell.org/ghc/  :? for help
Loading package ghc-prim ... linking ... done.
Loading package integer-gmp ... linking ... done.
Loading package base ... linking ... done.
[1 of 1] Compiling Main             ( sum.hs, interpreted )
Ok, modules loaded: Main.

ghci> prop_mySum 2 3
Loading package array-0.4.0.1 ... linking ... done.
Loading package deepseq-1.3.0.1 ... linking ... done.
Loading package old-locale-1.0.0.5 ... linking ... done.
Loading package time-1.4.0.1 ... linking ... done.
Loading package random-1.0.1.1 ... linking ... done.
Loading package containers-0.5.0.0 ... linking ... done.
Loading package pretty-1.1.1.0 ... linking ... done.
Loading package template-haskell ... linking ... done.
Loading package QuickCheck-2.6 ... linking ... done.
True

You can also invoke the quickCheck function in a main function, as shown below:

import Test.QuickCheck

mySum :: Int -> Int -> Int
mySum a b = a + b

prop_mySum :: Int -> Int -> Bool
prop_mySum a b = mySum a b == mySum b a

main :: IO ()
main = quickCheck prop_mySum

Compiling and executing the above code produces the following output:

$ ghc --make sum.hs
[1 of 1] Compiling Main             ( sum.hs, sum.o )
Linking sum ...

$ ./sum 
+++ OK, passed 100 tests.

You can also dump the input that was generated for the various test cases using the verboseCheck function, as shown below:

main :: IO ()
main = verboseCheck prop_mySum

Executing the above code with the updated main function will yield 100 input test cases that were generated in runtime.

ghci> main
Passed:
0
0
Passed:
-1
1
Passed:
64
-44
Passed:
-2159
2134
Passed:
-927480859
61832343
...

The head function in Haskell expects to receive a non-empty list. You can write a headExists function to check if the head exists for a list of integers, as shown below:

headExists :: [Int] -> Bool
headExists list
    | null list    = False
    | otherwise = True

You can load the above code in GHCi and test it out, as follows:

ghci> headExists []
False

ghci> headExists [1, 2, 3]
True

Let’s assume that, by mistake, you wrote an incorrect property-based test where the headExists function will always return ‘False’, ignoring the ‘otherwise’ case.

import Test.QuickCheck

headExists :: [Int] -> Bool
headExists list
    | null list    = False
    | otherwise = True

prop_headExists :: [Int] -> Bool
prop_headExists emptyList = headExists emptyList == False

main :: IO ()
main = quickCheck prop_headExists

Testing the code produces the following output:

$ ghc --make head.hs 
[1 of 1] Compiling Main             ( head.hs, head.o )
Linking head ...

$ ./head 
*** Failed! Falsifiable (after 3 tests):             
[0]

The QuickCheck library generated test cases for different [Int] types and it returned a failure after the third test, for which the input was [0]. Clearly, the ‘headExists [0]’ computation will return ‘True’ and not ‘False’.

The way we defined the property is incorrect. We know that if the list is empty, then its length is zero. We can write a helper function lengthZero for the above, as follows:

lengthZero :: [Int] -> Bool
lengthZero list
    | length list == 0 = True
    | otherwise = False

We can then use this function to assert that for any Integer list, if headExists returns ‘False’ then the lengthZero function must return ‘True’. The complete code is shown below:

import Data.List
import Test.QuickCheck

headExists :: [Int] -> Bool
headExists list
    | null list    = False
    | otherwise = True

lengthZero :: [Int] -> Bool
lengthZero list
    | length list == 0 = True
    | otherwise = False

prop_headExists :: [Int] -> Bool
prop_headExists list = headExists list == not (lengthZero list)

main :: IO ()
main = quickCheck prop_headExists

Executing the code produces the required output:

$ ghc --make head.hs 
[1 of 1] Compiling Main             ( head.hs, head.o )
Linking head ...

$ ./head 
+++ OK, passed 100 tests.

We can also re-write the above code based on conditional properties. The property that the headExists function will return ‘True’ only for non-empty lists can be defined as a constraint. The notation syntax is condition ==> property. In our example, if the condition that the list is non-empty is ‘True’, then the property that the headExists function for the list must return is ‘True’. Also, when the list is empty, the headExists function must return ‘False’. These two conditions can be written as follows:

import Data.List
import Test.QuickCheck

headExists :: [Int] -> Bool
headExists list
    | null list    = False
    | otherwise = True

prop_headExists :: [Int] -> Property
prop_headExists list = length list > 0 ==> headExists list == True

prop_emptyList :: [Int] -> Property
prop_emptyList list = length list == 0 ==> headExists list == False

main :: IO ()
main = do
     quickCheck prop_headExists
     quickCheck prop_emptyList

Testing the code produces the following output:

$ ghc --make cond.hs 
[1 of 1] Compiling Main             ( cond.hs, cond.o )
Linking cond ...

 $ ./cond 
+++ OK, passed 100 tests.
*** Gave up! Passed only 38 tests.

These tests can be integrated with Hspec or HUnit for a more verbose output.

Cabal

Cabal is a software tool that is used to describe a Haskell application, list its dependencies, and provide a manifestation to distribute the source and binaries. It is not to be confused with a distribution package manager like RPM or the Debian package management system. You can install Cabal using your distribution package manager. On Fedora, for example, you can use the following command:

$ sudo yum install cabal-install

Haskell software programs are available in hackage.haskell.org, and each project has a .cabal file. Let us take an example of the HSH-2.1.2 package at http://hackage.haskell.org/package/HSH which allows you to use shell commands and expressions within Haskell programs. You can download HSH-2.1.2.tar.gz and extract it using:

$ tar xzvf HSH-2.1.2.tar.gz 

HSH-2.1.2/
HSH-2.1.2/COPYING
HSH-2.1.2/HSH.cabal
HSH-2.1.2/testsrc/
HSH-2.1.2/testsrc/runtests.hs
HSH-2.1.2/HSH.hs
HSH-2.1.2/HSH/
HSH-2.1.2/HSH/Command.hs
HSH-2.1.2/HSH/ShellEquivs.hs
HSH-2.1.2/HSH/Channel.hs
HSH-2.1.2/COPYRIGHT
HSH-2.1.2/Setup.lhs

The .cabal file has various fields that describe the Haskell application. The contents of the HSH.cabal for version 2.1.2 are given below:

Name: HSH
Version: 2.1.3
License: LGPL
Maintainer: John Goerzen <jgoerzen@complete.org>
Author: John Goerzen
Stability: Beta
Copyright: Copyright (c) 2006-2014 John Goerzen
Category: system
license-file: COPYRIGHT
extra-source-files: COPYING
homepage: http://software.complete.org/hsh
Synopsis: Library to mix shell scripting with Haskell programs
Description: HSH is designed to let you mix and match shell expressions with
 Haskell programs. With HSH, it is possible to easily run shell
 commands, capture their output or provide their input, and pipe them
 to and from other shell commands and arbitrary Haskell functions at will.
  Category: System

Cabal-Version: >=1.2.3
Build-type: Simple

flag buildtests
  description: Build the executable to run unit tests
  default: False

library
  Exposed-Modules: HSH, HSH.Command, HSH.ShellEquivs, HSH.Channel
  Extensions: ExistentialQuantification, OverlappingInstances,
    UndecidableInstances, FlexibleContexts, CPP
  Build-Depends: base >= 4 && < 5, mtl, process, regex-compat, MissingH>=1.0.0,
    hslogger, filepath, regex-base, regex-posix, directory,
    bytestring
  if !os(windows)
    Build-Depends: unix
  GHC-Options: -O2 -threaded -Wall

Executable runtests
  if flag(buildtests)
    Buildable: True
    Build-Depends: base >= 4 && < 5, mtl, process, regex-compat,
      MissingH>=1.0.0,
      hslogger, filepath, regex-base, regex-posix, directory,
      bytestring, HUnit, testpack
    if !os(windows)
      Build-Depends: unix
  else
    Buildable: False
  Main-Is: runtests.hs
  HS-Source-Dirs: testsrc, .
  Extensions: ExistentialQuantification, OverlappingInstances,
    UndecidableInstances, FlexibleContexts, CPP
  GHC-Options: -O2 -threaded

Enter the HSH-2.1.2 directory and configure the project using the cabal configure command as shown below:

$ cd HSH-2.1.2

$ cabal configure
Resolving dependencies...
Configuring HSH-2.1.2...

You can then compile the project sources using the cabal build step:

$ cabal build

Building HSH-2.1.2...
Preprocessing library HSH-2.1.2...
[1 of 4] Compiling HSH.Channel      ( HSH/Channel.hs, dist/build/HSH/Channel.o )
...
[2 of 4] Compiling HSH.Command      ( HSH/Command.hs, dist/build/HSH/Command.o )
...
[3 of 4] Compiling HSH.ShellEquivs  ( HSH/ShellEquivs.hs, dist/build/HSH/ShellEquivs.o )
...
[4 of 4] Compiling HSH              ( HSH.hs, dist/build/HSH.o )
In-place registering HSH-2.1.2...

You can install the built library files using the cabal install command. By default, it installs to ~/.cabal folder as shown below:

$ cabal install

Resolving dependencies...
Configuring HSH-2.1.2...
Building HSH-2.1.2...
Preprocessing library HSH-2.1.2...
In-place registering HSH-2.1.2...
Installing library in /home/guest/.cabal/lib/HSH-2.1.2/ghc-7.6.3
Registering HSH-2.1.2...
Installed HSH-2.1.2

You can also generate HTML documentation for the source code using the cabal haddock option. The HTML files can also be made available at hackage.haskell.org.

$ cabal haddock

Running Haddock for HSH-2.1.2...
Preprocessing library HSH-2.1.2...
Warning: The documentation for the following packages are not installed. No
links will be generated to these packages: MissingH-1.3.0.1, rts-1.0,
hslogger-1.2.6, network-2.6.0.2
Haddock coverage:
...
Documentation created: dist/doc/html/HSH/index.html

If you make changes to the sources and wish to generate a new release, you can update the Version field in the HSH.cabal file.

Version: 2.1.3

In order to make a new tarball, use the cabal sdist command:

$ cabal sdist

Distribution quality warnings:
...
Building source dist for HSH-2.1.3...
Preprocessing library HSH-2.1.3...
Source tarball created: dist/HSH-2.1.3.tar.gz

To test the installed application, you can run GHCi from a directory other than the HSH-2.1.2 sources directory. For example:

$ ghci
GHCi, version 7.6.3: http://www.haskell.org/ghc/  :? for help
Loading package ghc-prim ... linking ... done.
Loading package integer-gmp ... linking ... done.
Loading package base ... linking ... done.

ghci> :m + HSH

ghci HSH> runIO "date"
Sun Jan  4 14:22:37 IST 2015

You should not run GHCi from the sources directory, since it will find the module in it and try to use it instead of the installed modules in ~/.cabal folder.

You can also test the installation by writing a program:

import HSH.Command

main :: IO ()
main = do
  runIO "date"

You can compile and execute the above as shown below:

$ ghc --make test.hs 
[1 of 1] Compiling Main             ( test.hs, test.o )
Linking test ...

$ ./test 
Sun Jan  4 14:25:19 IST 2015

You are encouraged to read the Cabal guide at https://www.haskell.org/cabal/ for information on specific fields and their options.

I attended PyCon India 2015 at NIMHANS Convention Centre, Bengaluru, India between October 2-4, 2015.

PyCon India 2015 stage

Day 1

I took an early train from Chennai to reach Bengaluru, and headed straight to the venue. The workshops were scheduled on the first day. After completing the registration formalities, I went to observe the DevSprint. I met a number of dgplug folks with whom I have been interacting online on #dgplug (irc.freenode.net).

After lunch, I attended Prof. Ronojoy Adhikari’s workshop on “Reasoning under uncertainity with Python”. Dorai Thodla began the workshop with an introduction on “Data Science” and its use and applications in the industry. Prof. Ronojoy then spoke on “Data Science: Theory” and “Probability Theory for Data Science”. We already use Boolean logic in our programming. We also have probability concepts implemented in Python. He then proceeded to demonstrate “Probabilistic Programming in Python” using Lea. He had presented a short preview of this talk at the Chennaipy August 2015 meet-up. Prof. Ronojoy showed very interesting Lea one-liners with IPython Notebook. A sample is provided below:

In [1]: from lea import *

In [2]: faircoin = Lea.fromVals('Head', 'Tail')

In [3]: faircoin.random(10)

Out[3]: ('Tail',
         'Tail',
         'Tail',
         'Head',
         'Head',
         'Tail',
         'Tail',
         'Head',
         'Tail',
         'Head')

In [4]: die = Lea.fromVals(1, 2, 3, 4, 5, 6)

In [5]: die1 = die.clone()
        die2 = die.clone()

In [6]: dice = die1 + die2

In [7]: dice

OUt[10]: 2 : 1/36
         3 : 2/36
         4 : 3/36
         5 : 4/36
         6 : 5/36
         7 : 6/36
         8 : 5/36
         9 : 4/36
        10 : 3/36
        11 : 2/36
        12 : 1/36

I had installed the Anaconda distribution and Lea for the workshop, but, couldn’t get Pomegranate to compile. Later, I worked with Jacob Schreiber to get it built on Ubuntu 14.10.

Prof. Ronojoy then explained Bayesian networks, and how the probabilities can be fed to compute the probability for a causal-effect relationship. He demonstrated numerous examples with live coding. His work is part of the Value from Data initiative. Prof. Ronojoy and Dorai’s “Bayesian Inference and Data Analytics” video lectures are available in YouTube.

Day 2

After collecting my speaker kit, I attended the “KG award” presentation, which was given to Vijay Kumar, followed by the key note by Dr. Ajit Kumar. Vijay Kumar is doing an excellent job in organizing the monthly Chennaipy meet-ups. It was also good to meet Dr. Ajit Kumar after many years, having known his work and involvement with the Phoenix project.

During the tea break, I took the time to test my laptop with the projector in auditorium-1, where I was scheduled to speak. I had few demos in my talk, and I needed to set everything up before the talk. After the tea break, I attended the session by Pratap Vardhan on “Consuming Government Data with Python and D3”. The insights from the data and the analysis by Gramener were quite interesting.

I then moved to the “dgplug” 1130 IST staircase meeting, where I met more #dgplug students, and was able to connect IRC nicknames to people’s faces. My suggestion to the students was to start to identify their interests, and to specialize on them. Around noon, I moved back to auditorium-1 to setup everything that I needed for my talk - “Pretty Printing in Python”. There is one slot which I never wanted - the one before lunch because I know that I will be the one standing between the audience and their lunch, and that is exactly what I got. Nevertheless, the slot had to be taken by someone, and I was prepared for it. I had requested the organizers for a whiteboard and markers for my talk, and they had arranged for the same. I had lot of material to cover in 45 minutes. I did rush through, but, paused at times to explain the concepts. The audience were quite interested in 3D printing, and they asked some really interesting questions at the end of the talk. The video is available in YouTube:

After a quick lunch, I headed for the lightning talks to present “Nursery Rhymes”. We are all familiar with nursery rhymes, and the idea is to teach programming languages and software development using them. The audience needed something to wake them after a heavy lunch, and they thoroughly enjoyed the singing! The video link is provided below:

I then attended the “Machine learning techniques for building a large scale production ready prediction system using Python” by Arthi Venkataraman. After a short tea break, I attended the talk by Pooja Salpekar on “Test Driven Development with Ansible”. She gave an overview of using Serverspec. The subject to test is still determined by the developer, and testing with Vagrant is not the same as testing in production deployments. For the last session of the day, I attended the “Python and Riak DB, a perfect couple for huge scale distributed computing” by Naren Arya, but, it was yet another generic talk.

In the evening, I participated in the discussion on building and running communities organized by Satyaakam Goswami on how Python communities are organizing themselves in India, and the challenges faced by them. I shared my experience with the Chennaipy meet-ups, and insights from my “i want 2 do project. tell me wat 2 do” presentation and book. A video recording of the discussion is available:

Day 3

Nicholas H. Tollervey began the day’s proceedings with his keynote. He shared his experience on Python and education. The Micro Python and BBC micro:bit projects are interesting initiatives. The audience did have questions on why they cannot use Android or any other mobile phone for Python development. But, I think they missed the point that Open Hardware initiatives are important and the openness is not the same with present day mobile devices. Nicholas also covered important points on how to organize Python Dojo meet-ups, and as a teacher, the experiences he shared were very useful.

After the tea break, I attended the session by Sumith Kulal on “SymEngine: The future fast core of computer algebra systems”. It was an introductory session on SymEngine, and he shared the roadmap and plan that they have for the software. I was then networking with people before attending “How to build microservices using ZeroMQ and WSGI” by Srinath G. S. It was yet another generic talk, and I headed for early lunch.

The previous night I prepared for another lightning talk titled “Yet Another Lightning Talk”, to once again, wake the audience after their heavy lunch. The talk went as I had planned, and the audience loved it! The video recording is available in YouTube:

A number of people approached me after the lightning talk, appreciating the creativity, and I did make a lot of new friends. The last talk of the day that I attended was on “Python load balancer; 0 to 1 million requests per second”, and yet again it turned out to be a a very generic talk, without going into specifics. I left early to catch a flight.

It will be helpful if organizers can identify talks where the speaker has contributed to the project. When you hire people for free and open source software (F/OSS) work, you would like to see what their contribution to the project is, and how they have added value to the project. I am interested to hear on what the speaker has done. I can always read the project web site, and find out more information about the project. I wish all conferences add this criteria in their talk selection process.

The workshops have been useful. The content of the talks need to be improved though. I was able to meet fellow F/OSS developers during the conference, and also made new friends. I am pleased that I was able to deliver a talk, and two lightning talks.

I would like to thank Manufacturing System Insights for sponsoring my travel, and allowing me to 3D print during office hours!

« OLDER POSTS