# PercyGrunwald's solution

## to Parallel Letter Frequency in the Elixir Track

Published at Jan 29 2019 · 0 comments
Instructions
Test suite
Solution

#### Note:

This exercise has changed since this solution was written.

Count the frequency of letters in texts using parallel computation.

Parallelism is about doing things in parallel that can also be done sequentially. A common example is counting the frequency of letters. Create a function that returns the total frequency of each letter in a list of texts and that employs parallelism.

## Running tests

Execute the tests with:

``````\$ elixir parallel_letter_frequency_test.exs
``````

### Pending tests

In the test suites, all but the first test have been skipped.

Once you get a test passing, you can unskip the next one by commenting out the relevant `@tag :pending` with a `#` symbol.

For example:

``````# @tag :pending
test "shouting" do
assert Bob.hey("WATCH OUT!") == "Whoa, chill out!"
end
``````

Or, you can enable all the tests by commenting out the `ExUnit.configure` line in the test suite.

``````# ExUnit.configure exclude: :pending, trace: true
``````

If you're stuck on something, it may help to look at some of the available resources out there where answers might be found.

## Submitting Incomplete Solutions

It's possible to submit an incomplete solution so you can see how others have completed the exercise.

### parallel_letter_frequency_test.exs

``````if !System.get_env("EXERCISM_TEST_EXAMPLES") do
end

ExUnit.start()
ExUnit.configure(exclude: :pending, trace: true)

# Your code should contain a frequency(texts, workers) function which accepts a
# list of texts and the number of workers to use in parallel.

defmodule FrequencyTest do
use ExUnit.Case

# Poem by Friedrich Schiller. The corresponding music is the European Anthem.
@ode_an_die_freude """
Freude schöner Götterfunken
Tochter aus Elysium,
Wir betreten feuertrunken,
Himmlische, dein Heiligtum!
Deine Zauber binden wieder
Was die Mode streng geteilt;
Alle Menschen werden Brüder,
Wo dein sanfter Flügel weilt.
"""

# Dutch national anthem
@wilhelmus """
Wilhelmus van Nassouwe
ben ik, van Duitsen bloed,
blijf ik tot in den dood.
Een Prinse van Oranje
ben ik, vrij, onverveerd,
den Koning van Hispanje
heb ik altijd geëerd.
"""

# American national anthem
@star_spangled_banner """
O say can you see by the dawn's early light,
What so proudly we hailed at the twilight's last gleaming,
Whose broad stripes and bright stars through the perilous fight,
O'er the ramparts we watched, were so gallantly streaming?
And the rockets' red glare, the bombs bursting in air,
Gave proof through the night that our flag was still there;
O say does that star-spangled banner yet wave,
O'er the land of the free and the home of the brave?
"""

# Returns the frequencies in a sorted list. This means it doesn't matter if
# your frequency() function returns a list of pairs or a map, the
# testing code will handle it.
defp freq(texts, workers \\ 4) do
Frequency.frequency(texts, workers) |> Enum.sort() |> Enum.into(%{})
end

# @tag :pending
test "no texts mean no letters" do
assert freq([]) == %{}
end

@tag :pending
test "one letter" do
assert freq(["a"]) == %{"a" => 1}
end

@tag :pending
test "case insensitivity" do
assert freq(["aA"]) == %{"a" => 2}
end

@tag :pending
test "many empty texts still mean no letters" do
assert freq(List.duplicate("  ", 10000)) == %{}
end

@tag :pending
test "many times the same text gives a predictable result" do
assert freq(List.duplicate("abc", 1000)) == %{"a" => 1000, "b" => 1000, "c" => 1000}
end

@tag :pending
test "punctuation doesn't count" do
assert freq([@ode_an_die_freude])[","] == nil
end

@tag :pending
test "numbers don't count" do
assert freq(["Testing, 1, 2, 3"])["1"] == nil
end

@tag :pending
test "all three anthems, together, 1 worker" do
freqs = freq([@ode_an_die_freude, @wilhelmus, @star_spangled_banner], 1)
assert freqs["a"] == 49
assert freqs["t"] == 56
assert freqs["ü"] == 2
end

@tag :pending
test "all three anthems, together, 4 workers" do
freqs = freq([@ode_an_die_freude, @wilhelmus, @star_spangled_banner], 4)
assert freqs["a"] == 49
assert freqs["t"] == 56
assert freqs["ü"] == 2
end
end``````
``````defmodule Frequency do
@doc """
Count letter frequency in parallel.

Returns a map of characters to frequencies.

The number of worker processes to use can be set with 'workers'.
"""
@spec frequency([String.t()], pos_integer) :: map
def frequency([], _workers), do: %{}

def frequency(texts, 1) do
texts
|> get_all_graphemes()
|> count_letters()
end

def frequency(texts, workers) do
texts
|> get_all_graphemes()
|> split_into_chunks(workers)
|> merge_results_stream()
end

defp split_into_chunks(all_graphemes, num_chunks) do
all_graphemes_count = Enum.count(all_graphemes)
graphemes_per_chunk = :erlang.ceil(all_graphemes_count / num_chunks)

Enum.chunk_every(all_graphemes, graphemes_per_chunk)
end

defp merge_results_stream(results_stream) do
Enum.reduce(results_stream, %{}, fn {:ok, worker_result}, acc ->
Map.merge(acc, worker_result, fn _key, acc_val, worker_val ->
acc_val + worker_val
end)
end)
end

defp get_all_graphemes(texts) do
texts
|> Enum.join()
|> String.graphemes()
end

defp count_letters(graphemes) do
Enum.reduce(graphemes, %{}, fn grapheme, acc ->
if String.match?(grapheme, ~r/^\p{L}\$/u) do
downcased_letter = String.downcase(grapheme)
Map.update(acc, downcased_letter, 1, fn count -> count + 1 end)
else
acc
end
end)
end
end``````