Avatar of joaofnds

joaofnds's solution

to ETL in the Common Lisp Track

Published at Oct 23 2019 · 0 comments
Instructions
Test suite
Solution

We are going to do the Transform step of an Extract-Transform-Load.

ETL

Extract-Transform-Load (ETL) is a fancy way of saying, "We have some crufty, legacy data over in this system, and now we need it in this shiny new system over here, so we're going to migrate this."

(Typically, this is followed by, "We're only going to need to run this once." That's then typically followed by much forehead slapping and moaning about how stupid we could possibly be.)

The goal

We're going to extract some scrabble scores from a legacy system.

The old system stored a list of letters per score:

  • 1 point: "A", "E", "I", "O", "U", "L", "N", "R", "S", "T",
  • 2 points: "D", "G",
  • 3 points: "B", "C", "M", "P",
  • 4 points: "F", "H", "V", "W", "Y",
  • 5 points: "K",
  • 8 points: "J", "X",
  • 10 points: "Q", "Z",

The shiny new scrabble system instead stores the score per letter, which makes it much faster and easier to calculate the score for a word. It also stores the letters in lower-case regardless of the case of the input letters:

  • "a" is worth 1 point.
  • "b" is worth 3 points.
  • "c" is worth 3 points.
  • "d" is worth 2 points.
  • Etc.

Your mission, should you choose to accept it, is to transform the legacy data format to the shiny new format.

Notes

A final note about scoring, Scrabble is played around the world in a variety of languages, each with its own unique scoring table. For example, an "E" is scored at 2 in the Māori-language version of the game while being scored at 4 in the Hawaiian-language version.

Setup

Check out Installing Common Lisp for instructions to get started or take a look at the guides available in the track's side bar.

Formatting

While Common Lisp doesn't care about indentation and layout of code, nor whether you use spaces or tabs, this is an important consideration for submissions to exercism.io. Excercism.io's code widget cannot handle mixing of tab and space characters well so using only spaces is recommended to make the code more readable to the human reviewers. Please review your editors settings on how to accomplish this. Below are instructions for popular editors for Common Lisp.

VIM

Use the following commands to ensure VIM uses only spaces for indentation:

:set tabstop=2
:set shiftwidth=2
:set expandtab

(or as a oneliner :set tabstop=2 shiftwidth=2 expandtab). This can be added to your ~/.vimrc file to use it all the time.

Emacs

Emacs is very well suited for editing Common Lisp and has many powerful add-on packages available. The only thing that one needs to do with a stock emacs to make it work well with exercism.io is to evaluate the following code:

(setq-default indent-tabs-mode nil)

This can be placed in your ~/.emacs (or ~/.emacs.d/init.el) in order to have it set whenever Emacs is launched.

One suggested add-on for Emacs and Common Lisp is SLIME which offers tight integration with the REPL; making iterative coding and testing very easy.

Source

The Jumpstart Lab team http://jumpstartlab.com

Submitting Incomplete Solutions

It's possible to submit an incomplete solution so you can see how others have completed the exercise.

etl-test.lisp

(ql:quickload "lisp-unit")
#-xlisp-test (load "etl")

(defpackage #:etl-test
  (:use #:common-lisp #:lisp-unit))

(in-package #:etl-test)

(defun make-hash (kvs)
  (reduce
   #'(lambda (h kv) (setf (gethash (first kv) h) (second kv)) h)
   kvs
   :initial-value (make-hash-table :test 'equalp)))

(define-test transform-one-value
  (assert-equalp (make-hash '(("world" 1)))
      (etl:transform (make-hash '((1 ("WORLD")))))))

(define-test transform-more-values
  (assert-equalp (make-hash '(("world" 1) ("gschoolers" 1)))
      (etl:transform (make-hash '((1 ("WORLD" "GSCHOOLERS")))))))

(define-test more-keys
  (assert-equalp (make-hash '(("apple" 1) ("artichoke" 1) ("boat" 2) ("ballerina" 2)))
      (etl:transform (make-hash '((1 ("APPLE" "ARTICHOKE")) (2 ("BOAT" "BALLERINA")))))))

(define-test full-dataset
  (let ((input-data '((1 ("A" "E" "I" "O" "U" "L" "N" "R" "S" "T"))
                      (2 ("D" "G"))
                      (3 ("B" "C" "M" "P"))
                      (4 ("F" "H" "V" "W" "Y"))
                      (5 ("K"))
                      (8 ("J" "X"))
                      (10 ("Q" "Z"))))
        (expected-output '(("a" 1) ("b" 3) ("c" 3) ("d" 2) ("e" 1)
                           ("f" 4) ("g" 2) ("h" 4) ("i" 1) ("j" 8)
                           ("k" 5) ("l" 1) ("m" 3) ("n" 1) ("o" 1)
                           ("p" 3) ("q" 10) ("r" 1) ("s" 1) ("t" 1)
                           ("u" 1) ("v" 4) ("w" 4) ("x" 8) ("y" 4)
                           ("z" 10))))
    (assert-equalp (make-hash expected-output)
        (etl:transform (make-hash input-data)))))

#-xlisp-test
(let ((*print-errors* t)
      (*print-failures* t))
  (run-tests :all :etl-test))
(in-package #:cl-user)
(defpackage #:etl
  (:use #:common-lisp)
  (:export #:transform))

(in-package #:etl)

(defun transform (data)
  (let ((new-point-format (make-hash-table :test #'equalp)))
    (maphash
     (lambda (points letters)
       (dolist (letter letters)
         (setf (gethash letter new-point-format) points)))
     data)
    new-point-format))

Community comments

Find this solution interesting? Ask the author a question to learn more.

What can you learn from this solution?

A huge amount can be learned from reading other people’s code. This is why we wanted to give exercism users the option of making their solutions public.

Here are some questions to help you reflect on this solution and learn the most from it.

  • What compromises have been made?
  • Are there new concepts here that you could read more about to improve your understanding?