Lectures/Week-06-ReadingFiles.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Week 6: Reading Files!\n",
    "\n",
    "Now we start getting into the good stuff.\n",
    "\n",
    "This week we're learning the basics of handling files for reading in and writing out.  We're used to our operating systems handling all these things for us, so we need to reproduce those backend mechanisms ourselves.  This is a topic where even they very high level language of python starts feeling very low level (closer to computers than human language).  As you start out, you may have to rely on just rotely reproducing these patterns.  So long as you know how they are operating, you can reasonably move forward without really understanding what is happening.  That said, this can be a dangerous game when you are writing files.\n",
    "\n",
    "You will find that there are 3 main patterns for reading in files and processing the data.  These 3 are core to python and you will find that they are roughly interchangable.  Not that they do the same things, but that you can almost always accomplish whatever you need to do with the file contents no matter which pattern you choose to use.  \n",
    "Nearly all students find that they have a favorite.  This week's homework forces you to use all three, because sometimes your go-to pattern isn't the best.  But moving forward, you should be able to use your favorite most of the times.  The division usually happens between readline() and readlines()/read().  Much like knitting and crochet, one side usually finds the other to be unfathomable.\n",
    "\n",
    "You'll have to fuss at things in different ways, but don't worry about trying to use the 'right' one.  I suggest that you use the one you understand the most and go from there.  As you gain more python experience you'll get more comfortable and start adding in more.\n",
    "\n",
    "Unless the assignment names a specific method to be used, you should use the method that you are the most comfortable with.\n",
    "\n",
    "While there are 3 basic methods of reading in files, each is going to have roughly the same framework:\n",
    "\n",
    "1. Open the file in whichever mode (reading or writing) you need \n",
    "    * This will be with `file_in = open(file_path, mode)` (note that this is just the pattern to use, you'll have to supply the file path and mode values yourself)\n",
    "    * Remember that you will always have to do an assignment statement to save the file object so you can act on it later.  I'm using `file_in` as an example variable name, but you can change it.\n",
    "    * This creates you file IO object you will then mess with in step 2\n",
    "    * you may need to have `open(file_path, mode, encoding = 'utf-8')` if your computer is based in another lanugage.\n",
    "2. Do stuff to that file IO object\n",
    "    * This is where each well be really different, but you'll be doing various actions to your `file_in` object, or whatever the variable name of your file IO object is.\n",
    "3. Close the file.\n",
    "    * this will always be the same pattern, `file_in.close()`. (replace `file_in` with your file IO variable name)\n",
    "\n",
    "There are ways of handling files in sort of a one shot, but we'll get to that later.\n",
    "\n",
    "An important thing to keep in mind:  the variable that you create with your `open()` expression will be your access point to the content of that file.  The content within that file is accessed by that object at your direction.  That is the purpose of this object.  This object is how python knows about your file.  But this doesn't mean that the file IO object knows about your file's content.  You'll need to direct that object to access or write the content that you want.\n",
    "\n",
    "# Some other cautions \n",
    "\n",
    "Just a word of warning, there are some packages (like pandas) have their own methods for reading in files that take care of all these things for you, so you won't see the use of `open()`, `.close()`, or any other functions for reading the file contents.\n",
    "\n",
    "You should also only be using plain text files for the file reading methods that we will be discussing here.  They will not work with propietary files like Word or Excel documents.  This is why certain packages have their own functions, as I mentioned just before.  They are equipped with additional tools to read those kinds of files.\n",
    "\n",
    "# Once you have that file object\n",
    "\n",
    "There are some structural considerations to keep in mind.  Some reading methods bypass having to deal with this stuff, but these things will still be happening even if you can get away without having to think about them.\n",
    "\n",
    "# The line, revisited\n",
    "\n",
    "We've spent several of our class sessions talking about the importance of lines in data processing.  Processing files is one of those reasons that we have focused on it.  Several of the file reading using navigates through the content based on newlines.  Newlines are really one of the most essential units of data.\n",
    "\n",
    "## The cursor\n",
    "\n",
    "Don't curse the cursor!  When you open a file for reading, there's an invisible cursor that the system places within that file.  It starts at the very beginning of the file and moves forward at your direction.  This means you can direct it to go forward character by character or line by line.  But you can never move backwards!  This means that you cannot 'reread' a file once you've already done a read action to it.  Once the cursor has found the end of the file, it stops and will not move forward unless it receives a command to do so.\n",
    "\n",
    "For example:\n",
    "\n",
    "1. You open a file for reading\n",
    "2. You use `.read()` on that file.  The cursor is now at the end of the file.\n",
    "3. You cannot then use `.readlines()` on it after because `.readlines()` is going to attempt to start reading from the end of that file.  You'll get nothing.\n",
    "\n",
    "Just as we need to take some time to think about how we might want to loop over something, once you're accustomed to how these different methods interact with data, you can take some time to think about how you want to maneuver the cursor over the file that you are working with.  "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# The 3 essential patterns:  `.read()`, `.readlines()`, and `.readline()`\n",
    "\n",
    "We're going to cover these in order of complexity.\n",
    "\n",
    "1. `.read()` reads the entire contents of the file into a single string, that you can then operate on like any normal string.  You will have a very short file reading section in your program.  This is often the best one to get started with, because we've been used to having all our text and data as one big string. \n",
    "\n",
    "2. `.readlines()` will read an entire file into a list of the lines in the file, so each line in the file will become an element in that list.  This preserves all the order and content of the file, but gives it back to you in a handy structure of a list.  This in valuable, of course, when you want a list of lines and effectively is like running `.split('\\n')` on your file contents.  This helps you skip a step if that is your end goal.\n",
    "\n",
    "3. `.readline()` will read the file line by line starting at the beginning.  This is the one where the cursor is something that you are directly interacting with, as it will only move forward within the file at your direction.\n",
    "\n",
    "## But why deal with files anyhow?\n",
    "\n",
    "\n",
    "The other benefit is that the script and code can be made to be independent of the file.  For example, you may have a report to write each month on website log metricts.  Each month you receive a data file from a colleague, the server, or downloaded from something like google analytics.  The contents of the data file obviously change each month, but the structure does not.  This means that you can write your analysis script once and rerun it over the file that you recieve each month.  You may need to change parameters along the way, but the hard work is done.\n",
    "\n",
    "\n",
    "## `.read()`: read all the things into one string\n",
    "\n",
    "When usure about which method to use, use `.read()`\n",
    "\n",
    "This is the basic formula for `.read()`, which will read the entire contents of the file into a single string.  We encountered this last week when I showed how we read in that data file all at once.  After you have read the file contents into that string, you will operate on that string instead of the file object.  The nice thing is now that the contents are a regular string, so you can use all your normal string operations on it.  \n",
    "\n",
    "In fact, this is the point of this kind of thing.  The program reads the data into active memory, then does stuff to it.  This allows you to have a very large data file live outside of your script, perhaps even an another server or database.\n",
    "\n",
    "\n",
    "The formula for `.read()` is one of the simplest, and this is usually done at the beginning of your program.\n",
    "\n",
    "* Step 1\n",
    "    * `fileIOvariable = open(filepath, mode)`\n",
    "* Step 2\n",
    "    * `file_contents = fileIOvariable.read()`\n",
    "* Step 3\n",
    "    * `fileIOvariable.close()`"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "This is all the text from the .read():\n",
      "---------------------!!!!!!!!!\n",
      "Hello, I would like to science.\n",
      "Please show me where the science is.\n",
      "I am a meat popcicle.\n"
     ]
    }
   ],
   "source": [
    "my_file = open('smalltext.txt', 'rt') # 1\n",
    "\n",
    "all_the_text = my_file.read() # 2\n",
    "\n",
    "print(\"This is all the text from the .read():\")\n",
    "print(\"---------------------!!!!!!!!!\")\n",
    "print(all_the_text) # just the first 10 lines, please\n",
    "\n",
    "my_file.close() # 3"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "That is it.  At this point, all the data that you want is inside of `all_the_text`.  After the `.read()` has been executed the cursor is at the end of the file.  There is nothing left to read, so when you ask it to read the file again, there is no more text to traverse over so you get back an empty string.\n",
    "\n",
    "We can see `.read()` in action when this happens."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "This is all the text from the FIRST .read():\n",
      "---------------------\n",
      "Hello, I would like to science.\n",
      "Please show me where the science is.\n",
      "I am a meat popcicle.\n",
      "---------------------\n",
      "This is all the text from the SECOND .read():\n",
      "---------------------\n",
      "\n"
     ]
    }
   ],
   "source": [
    "my_file = open('smalltext.txt', 'rt')\n",
    "\n",
    "all_the_text = my_file.read()\n",
    "\n",
    "print(\"This is all the text from the FIRST .read():\")\n",
    "print(\"---------------------\")\n",
    "print(all_the_text) # just the first 10 lines, please\n",
    "\n",
    "# now the cursor is at the end of the text in the file\n",
    "# we can still use .read(), but since it is at the end of the file we get an empty string back\n",
    "\n",
    "maybe_more = my_file.read() # here's the second file read attempt\n",
    "\n",
    "print(\"---------------------\")\n",
    "print(\"This is all the text from the SECOND .read():\")\n",
    "print(\"---------------------\")\n",
    "print(maybe_more) # an empty string\n",
    "\n",
    "my_file.close()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Printing these out sort of hide what's happening in these strings. So let's use some handy interactive stuff.\n",
    "\n",
    "The `all_the_text` variable contains the contents of our first `.read()` call, so it has all the contents of our file.\n",
    "\n",
    "The `maybe_more` variable contains whatever was returned from our second `.read()` call, so we can see better now that it is an empty string."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "'Hello, I would like to science.\\nPlease show me where the science is.\\nI am a meat popcicle.'"
      ]
     },
     "execution_count": 2,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "all_the_text"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "''"
      ]
     },
     "execution_count": 3,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "maybe_more"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "You can no longer do reading or writing operations to a file IO object that has been closed (so `.close()` has been called on it.\n",
    "\n",
    "You will generate an error if you try to do that.  I'm adapting our previous example to show what happens when `.read()` is called on our file object after `.close()` has been called on it.  This generates a pretty helpful error message, saying that \"`I/O operation on closed file.`\""
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "This is all the text from the FIRST .read():\n",
      "---------------------\n",
      "Hello, I would like to science.\n",
      "Please show me where the science is.\n",
      "I am a meat popcicle.\n"
     ]
    },
    {
     "ename": "ValueError",
     "evalue": "I/O operation on closed file.",
     "output_type": "error",
     "traceback": [
      "\u001b[0;31m---------------------------------------------------------------------------\u001b[0m",
      "\u001b[0;31mValueError\u001b[0m                                Traceback (most recent call last)",
      "\u001b[0;32m<ipython-input-4-9826284dd2c1>\u001b[0m in \u001b[0;36m<module>\u001b[0;34m()\u001b[0m\n\u001b[1;32m      9\u001b[0m \u001b[0mmy_file\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mclose\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m     10\u001b[0m \u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0;32m---> 11\u001b[0;31m \u001b[0mmaybe_more\u001b[0m \u001b[0;34m=\u001b[0m \u001b[0mmy_file\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mread\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0m",
      "\u001b[0;31mValueError\u001b[0m: I/O operation on closed file."
     ]
    }
   ],
   "source": [
    "my_file = open('smalltext.txt', 'rt')\n",
    "\n",
    "all_the_text = my_file.read()\n",
    "\n",
    "print(\"This is all the text from the FIRST .read():\")\n",
    "print(\"---------------------\")\n",
    "print(all_the_text) # just the first 10 lines, please\n",
    "\n",
    "my_file.close()\n",
    "\n",
    "maybe_more = my_file.read()\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## `.readlines()` note the plural\n",
    "\n",
    "Like `.read()`, this read method will read the entire file's contents.  Instead of getting a string containing all the contents, you'll get a list with all the contents split up on lines.\n",
    "\n",
    "Note that interestingly enough, this will split all the lines up after the newline, but unlike using `.split('\\n')`, the newline characters will be retained.\n",
    "\n",
    "The formula:\n",
    "\n",
    "* Step 1\n",
    "    * `fileIOvariable = open(filepath, mode)`\n",
    "    * This is the same as before\n",
    "* Step 2\n",
    "    * `file_contents_list = fileIOvariable.readlines()`\n",
    "    * This is pretty much the same as before, but calling `.readlines()` instead of `.read()`\n",
    "* Step 3\n",
    "    * `fileIOvariable.close()`\n",
    "    * this is the same as before"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "my_file = open('smalltext.txt', 'rt') # 1\n",
    "\n",
    "contents_list = my_file.readlines() # 2\n",
    "\n",
    "my_file.close()  # 3"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "['Hello, I would like to science.\\n', 'Please show me where the science is.\\n', 'I am a meat popcicle.']\n"
     ]
    }
   ],
   "source": [
    "print(contents_list)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "As mentioned before, this is a method of convienience.  If you want to end up with a list of the lines you can do that directly with this method.  The usage pattern is the same as `.read()` but you are calling `.readlines()` instead.\n",
    "\n",
    "Honestly, that's it."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## `.readline()` note the singular here"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "So `maybemore` worked, but is empty.  The cursor had no more text to go through, so it just gave us an empty string.  But let's look at this a different way and see if we can halt the cursor in the middle of a file."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "A told B, and B told C, \"I'll meet you at the top of the coconut tree.\"\n",
      "\n",
      "\"Wheee!\" said D to E F G, \"I'll beat you to the top of the coconut tree.\"\n",
      "\n",
      "Chicka chicka boom boom! Will there be enough room? Here comes H up the coconut tree,\n",
      "\n",
      "and I and J and tag-along K, all on their way up the coconut tree.\n",
      "\n",
      "Chicka chicka boom boom! Will there be enough room? Look who's coming! L M N O P!\n",
      "\n"
     ]
    }
   ],
   "source": [
    "my_file = open('boomboom.txt', 'rt')\n",
    "\n",
    "for justdothis5times in range(5):\n",
    "    print(my_file.readline())\n",
    "    \n",
    "therest = my_file.read()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "And Q R S! And T U V! Still more - W! And X Y Z!\n",
      "The whole alphabet up the - Oh, no! Chicka chicka... BOOM! BOOM!\n",
      "Skit skat skoodle doot. Flip flop flee. Everybody running to the coconut tree.\n",
      "Mamas and papas and uncles and aunts hug their little dears, then dust their pants.\n",
      "\"Help us up,\" cried A B C.\n",
      "Next from the pileup skinned-knee D and stubbed-toe E and patched-up F. Then comes G all out of breath.\n",
      "H is tangled up with I. J and K are about to cry. L is knotted like a tie.\n",
      "M is looped. N is stopped. O is twisted alley-oop. Skit skat skoodle doot. Flip flop flee.\n",
      "Look who's coming! It's black-eyed P, Q R S, and loose-tooth T. Then U V W wiggle-jiggle free.\n",
      "Last to come X Y Z. And the sun goes down on the coconut tree...\n",
      "But - chicka chicka boom boom! Look, there's a full moon.\n",
      "A is out of bed, and this is what he said, \"Dare double dare, you can't catch me.\n",
      "Chicka chicka BOOM! BOOM!Chicka chicka BOOM! BOOM!\n",
      "I'll beat you to the top of the coconut tree.\"\n",
      "Chicka chicka BOOM! BOOM!\n"
     ]
    }
   ],
   "source": [
    "print(therest)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "What happened here?  We can see that the `.readline()` bit grabbed the first 5 lines and then the `.read()` got the rest of the lines.  There also seems to be extra newlines happening in the first section?\n",
    "\n",
    "I used a for loop with `range(5)` to repeat `.readline()` 5 times.  This meant it acted 5 independent times, so it printed out 5 lines that each ended in `\\n`, which means that gets rendered as an extra newline, because there's no text right after it.\n",
    "\n",
    "At this point, the cursor is sitting at the beginning of line 6 just waiting.  When I call `.read()` it goes through the remaining portion of the file.  There's no extra newlines happening because there's text to sit in those slots."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# A problem to work through\n",
    "\n",
    "(editor's note: I kind of hate this example and usually skip this, but don't want to trash it just yet).\n",
    "\n",
    "Here's something silly, but it's a task.\n",
    "\n",
    "Read through a text file and change it so that the lines alternate between upper and lower case (starting with upper), then write out the new file. \n",
    "\n",
    "So we want something like this:\n",
    "```\n",
    "LINE 1\n",
    "line 2\n",
    "LINE 3\n",
    "line 4\n",
    "```\n",
    "\n",
    "Let's break this down:\n",
    "\n",
    "1. Read in the file\n",
    "2. Transform the text\n",
    "3. Write out the file\n",
    "\n",
    "We can go ahead and set up items 1 and 3."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "file_in = open('boomboom.txt', 'rt')\n",
    "\n",
    "write_me = file_in.read()\n",
    "\n",
    "file_out = open('newboom.txt', 'wt')\n",
    "\n",
    "file_out.write(write_me)\n",
    "\n",
    "file_in.close()\n",
    "file_out.close()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "In the code in the cell just above, we can see that we aren't transforming the text yet, but we are at leat set up for the read in and write out.  While we're playing with things, let's comment out the write stuff and just use print statements."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "A told B, and B told C, \"I'll meet you at the top of the coconut tree.\"\n",
      "\"Wheee!\" said D to E F G, \"I'll beat you to the top of the coconut tree.\"\n",
      "Chicka chicka boom boom! Will there be enough room? Here comes H up the coconut tree,\n",
      "and I and J and tag-along K, all on their way up the coconut tree.\n",
      "Chicka chicka boom boom! Will there be enough room? Look who's coming! L M N O P!\n",
      "And Q R S! And T U V! Still more - W! And X Y Z!\n",
      "The whole alphabet up the - Oh, no! Chicka chicka... BOOM! BOOM!\n",
      "Skit skat skoodle doot. Flip flop flee. Everybody running to the coconut tree.\n",
      "Mamas and papas and uncles and aunts hug their little dears, then dust their pants.\n",
      "\"Help us up,\" cried A B C.\n",
      "Next from the pileup skinned-knee D and stubbed-toe E and patched-up F. Then comes G all out of breath.\n",
      "H is tangled up with I. J and K are about to cry. L is knotted like a tie.\n",
      "M is looped. N is stopped. O is twisted alley-oop. Skit skat skoodle doot. Flip flop flee.\n",
      "Look who's coming! It's black-eyed P, Q R S, and loose-tooth T. Then U V W wiggle-jiggle free.\n",
      "Last to come X Y Z. And the sun goes down on the coconut tree...\n",
      "But - chicka chicka boom boom! Look, there's a full moon.\n",
      "A is out of bed, and this is what he said, \"Dare double dare, you can't catch me.\n",
      "Chicka chicka BOOM! BOOM!Chicka chicka BOOM! BOOM!\n",
      "I'll beat you to the top of the coconut tree.\"\n",
      "Chicka chicka BOOM! BOOM!\n"
     ]
    }
   ],
   "source": [
    "file_in = open('boomboom.txt', 'rt')\n",
    "\n",
    "write_me = file_in.read()\n",
    "\n",
    "\n",
    "print(write_me)\n",
    "# file_out = open('newboom.txt', 'wt')\n",
    "\n",
    "# file_out.write(write_me)\n",
    "\n",
    "file_in.close()\n",
    "# file_out.close()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Great! Now let's think of an approach for how to do this.  We know that we can use `.upper()` and `.lower()` to transform the text, but the issue is: how do we alternate lines of text?\n",
    "\n",
    "Think of it this way: we need to move through the file in terms of pairs.  One upper, one lower then one upper and one lower.\n",
    "\n",
    "We can use the cursor to our advantage here.  If we're already iterating through reading a file using a for loop, e.g.\n",
    "\n",
    "```python\n",
    "for line in file_in:\n",
    "    print(line)\n",
    "\n",
    "```\n",
    "\n",
    "We're just using this for loop as a convenient way to move the cursor one by one.  So can we move thorugh two lines within that for loop?  We can't just use line twice becuase it'll always be the second line, but we have another method in our pockets:  `.readline()`.  As we saw with our `range(5)` example, we can call `.readline()` an arbitrary number of times.  It won't mess up our for loop, but it will give us the next line, and push the cursor forward by one.\n",
    "\n",
    "In other words:\n",
    "\n",
    "\n",
    "```python\n",
    "\n",
    "# let's say the cursor starts at 0\n",
    "\n",
    "for line in file_in: # cursor + 1; cursor is now 1\n",
    "    print(line)      # cursor doesn't change; cursor remains 1\n",
    "    print(file_in.readline()) # cursor + 1; cursor is now 2\n",
    "    \n",
    "# now this will run again\n",
    "\n",
    "# ...\n",
    "\n",
    "# cursor is at 2 from the previous loop \n",
    "\n",
    "for line in file_in: # cursor + 1; cursor is now 3\n",
    "    print(line)      # cursor doesn't change; cursor remains 3\n",
    "    print(file_in.readline()) # cursor + 1; cursor is now 4\n",
    "    \n",
    "# and so on\n",
    "```"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "collapsed": true
   },
   "source": [
    "Let's play with this using just a regular counter and an accumulator pattern.  "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "[1, 1, 1, 1, 1, 1, 1, 1, 1, 1]\n",
      "2\n",
      "4\n",
      "6\n",
      "8\n",
      "10\n",
      "12\n",
      "14\n",
      "16\n",
      "18\n",
      "20\n"
     ]
    }
   ],
   "source": [
    "total = 0\n",
    "\n",
    "lotsofones = [1] * 10\n",
    "print(lotsofones)\n",
    "\n",
    "for one in lotsofones:\n",
    "    total = total + one\n",
    "    total = total + 1\n",
    "    print(total)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "collapsed": true
   },
   "source": [
    "So plunking all of this in together:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 10,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "A TOLD B, AND B TOLD C, \"I'LL MEET YOU AT THE TOP OF THE COCONUT TREE.\"\n",
      "\"wheee!\" said d to e f g, \"i'll beat you to the top of the coconut tree.\"\n",
      "CHICKA CHICKA BOOM BOOM! WILL THERE BE ENOUGH ROOM? HERE COMES H UP THE COCONUT TREE,\n",
      "and i and j and tag-along k, all on their way up the coconut tree.\n",
      "CHICKA CHICKA BOOM BOOM! WILL THERE BE ENOUGH ROOM? LOOK WHO'S COMING! L M N O P!\n",
      "and q r s! and t u v! still more - w! and x y z!\n",
      "THE WHOLE ALPHABET UP THE - OH, NO! CHICKA CHICKA... BOOM! BOOM!\n",
      "skit skat skoodle doot. flip flop flee. everybody running to the coconut tree.\n",
      "MAMAS AND PAPAS AND UNCLES AND AUNTS HUG THEIR LITTLE DEARS, THEN DUST THEIR PANTS.\n",
      "\"help us up,\" cried a b c.\n",
      "NEXT FROM THE PILEUP SKINNED-KNEE D AND STUBBED-TOE E AND PATCHED-UP F. THEN COMES G ALL OUT OF BREATH.\n",
      "h is tangled up with i. j and k are about to cry. l is knotted like a tie.\n",
      "M IS LOOPED. N IS STOPPED. O IS TWISTED ALLEY-OOP. SKIT SKAT SKOODLE DOOT. FLIP FLOP FLEE.\n",
      "look who's coming! it's black-eyed p, q r s, and loose-tooth t. then u v w wiggle-jiggle free.\n",
      "LAST TO COME X Y Z. AND THE SUN GOES DOWN ON THE COCONUT TREE...\n",
      "but - chicka chicka boom boom! look, there's a full moon.\n",
      "A IS OUT OF BED, AND THIS IS WHAT HE SAID, \"DARE DOUBLE DARE, YOU CAN'T CATCH ME.\n",
      "chicka chicka boom! boom!chicka chicka boom! boom!\n",
      "I'LL BEAT YOU TO THE TOP OF THE COCONUT TREE.\"\n",
      "chicka chicka boom! boom!\n"
     ]
    }
   ],
   "source": [
    "file_in = open('boomboom.txt', 'rt')\n",
    "\n",
    "new_text = []\n",
    "\n",
    "for line in file_in:\n",
    "    new_text.append(line.upper())\n",
    "    new_text.append(file_in.readline().lower())\n",
    "\n",
    "write_me = \"\".join(new_text) # I retained the newlines as they came in from the original file\n",
    "                             # so I don't need to put them back in\n",
    "\n",
    "print(write_me)\n",
    "# file_out = open('newboom.txt', 'wt')\n",
    "\n",
    "# file_out.write(write_me)\n",
    "\n",
    "# file_in.close()\n",
    "# file_out.close()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 27,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "'A TOLD B, AND B TOLD C, \"I\\'LL MEET YOU AT THE TOP OF THE COCONUT TREE.\"\\n\"wheee!\" said d to e f g, \"i\\'ll beat you to the top of the coconut tree.\"\\nCHICKA CHICKA BOOM BOOM! WILL THERE BE ENOUGH ROOM? HERE COMES H UP THE COCONUT TREE,\\nand i and j and tag-along k, all on their way up the coconut tree.\\nCHICKA CHICKA BOOM BOOM! WILL THERE BE ENOUGH ROOM? LOOK WHO\\'S COMING! L M N O P!\\nand q r s! and t u v! still more - w! and x y z!\\nTHE WHOLE ALPHABET UP THE - OH, NO! CHICKA CHICKA... BOOM! BOOM!\\nskit skat skoodle doot. flip flop flee. everybody running to the coconut tree.\\nMAMAS AND PAPAS AND UNCLES AND AUNTS HUG THEIR LITTLE DEARS, THEN DUST THEIR PANTS.\\n\"help us up,\" cried a b c.\\nNEXT FROM THE PILEUP SKINNED-KNEE D AND STUBBED-TOE E AND PATCHED-UP F. THEN COMES G ALL OUT OF BREATH.\\nh is tangled up with i. j and k are about to cry. l is knotted like a tie.\\nM IS LOOPED. N IS STOPPED. O IS TWISTED ALLEY-OOP. SKIT SKAT SKOODLE DOOT. FLIP FLOP FLEE.\\nlook who\\'s coming! it\\'s black-eyed p, q r s, and loose-tooth t. then u v w wiggle-jiggle free.\\nLAST TO COME X Y Z. AND THE SUN GOES DOWN ON THE COCONUT TREE...\\nbut - chicka chicka boom boom! look, there\\'s a full moon.\\nA IS OUT OF BED, AND THIS IS WHAT HE SAID, \"DARE DOUBLE DARE, YOU CAN\\'T CATCH ME.\\nchicka chicka boom! boom!chicka chicka boom! boom!\\nI\\'LL BEAT YOU TO THE TOP OF THE COCONUT TREE.\"\\nchicka chicka boom! boom!'"
      ]
     },
     "execution_count": 27,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": []
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "collapsed": true
   },
   "source": [
    "# Another problem statement\n",
    "\n",
    "Your friend says to you that there's a secret code in the Raven, where the first word of each line can go together to make a snarky political statement.  So you decide to write a python script that goes through each line and captures the first word of each line.\n",
    "\n",
    "## Where to start?\n",
    "\n",
    "We need to think of the things that we know we can do.  \n",
    "\n",
    "1. our ability to loop over lines in a text file\n",
    "2. our ability to split a line of text up into words\n",
    "3. our ability to get the first word out of that list of words\n",
    "\n",
    "We can tackle these one at a time.  We may not have done all these things all together, but we have done them all separately.  Let's remind ourselves of how we do this from the smallest to the largest."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Get the first thing out of a list\n",
    "\n",
    "Remember our positions, right?  Given a list called `stuff`, no matter what's in there, so long as it has at least one thing in there we can say `stuff[0]` to get that thing out of it."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "1\n"
     ]
    }
   ],
   "source": [
    "stuff1 = [1, 2, 3]\n",
    "print(stuff1[0])"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "a\n"
     ]
    }
   ],
   "source": [
    "stuff2 = ['a', 'b', 'c']\n",
    "print(stuff2[0])"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Let's also take the time to remind ourselves what it looks like when there's nothing in there."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "metadata": {},
   "outputs": [
    {
     "ename": "IndexError",
     "evalue": "list index out of range",
     "output_type": "error",
     "traceback": [
      "\u001b[0;31m--------------------------------------------------\u001b[0m",
      "\u001b[0;31mIndexError\u001b[0m       Traceback (most recent call last)",
      "\u001b[0;32m<ipython-input-6-a86258252b4a>\u001b[0m in \u001b[0;36m<module>\u001b[0;34m()\u001b[0m\n\u001b[1;32m      1\u001b[0m \u001b[0mstuff3\u001b[0m \u001b[0;34m=\u001b[0m \u001b[0;34m[\u001b[0m\u001b[0;34m]\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0;32m----> 2\u001b[0;31m \u001b[0mprint\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mstuff3\u001b[0m\u001b[0;34m[\u001b[0m\u001b[0;36m0\u001b[0m\u001b[0;34m]\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0m",
      "\u001b[0;31mIndexError\u001b[0m: list index out of range"
     ]
    }
   ],
   "source": [
    "stuff3 = []\n",
    "print(stuff3[0])"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Making a list of words\n",
    "\n",
    "Remember that we don't care about the punctuation here, let's just use the whitespace."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "['I', 'am', 'a', 'meat', 'popsicle.']\n"
     ]
    }
   ],
   "source": [
    "sentence = \"I am a meat popsicle.\"\n",
    "words = sentence.split()\n",
    "print(words)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 9,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "I\n"
     ]
    }
   ],
   "source": [
    "# and putting it together\n",
    "\n",
    "print(words[0])"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Loop over a list of lines in a file\n",
    "\n",
    "We know several ways to get lines out of a document.  We can use `.readlines()`, we can split up a string we read in from `.read()`, and a few other things.  Let's use a pattern that we can see on page 157.\n",
    "\n",
    "There's a lot of shorthand so that we don't need to use `.readline()`.  This difference can make what's happening quite difficult to follow.\n",
    "\n",
    "Let's start on our copy of the raven.  We've got a lot going on here, so this is where clear and correct variable names come in handy.\n",
    "\n",
    "**Even though you might be copying a formula from the book, you really must change the variable names for you own purpose.**"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 15,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "['Once', 'upon', 'a', 'midnight', 'dreary,', 'while', 'I', 'pondered,', 'weak', 'and', 'weary,']\n",
      "['Over', 'many', 'a', 'quaint', 'and', 'curious', 'volume', 'of', 'forgotten', 'lore—']\n",
      "['While', 'I', 'nodded,', 'nearly', 'napping,', 'suddenly', 'there', 'came', 'a', 'tapping,']\n",
      "['As', 'of', 'some', 'one', 'gently', 'rapping,', 'rapping', 'at', 'my', 'chamber', 'door.']\n",
      "['“’Tis', 'some', 'visitor,”', 'I', 'muttered,', '“tapping', 'at', 'my', 'chamber', 'door—']\n",
      "['Only', 'this', 'and', 'nothing', 'more.”']\n",
      "[]\n",
      "['Ah,', 'distinctly', 'I', 'remember', 'it', 'was', 'in', 'the', 'bleak', 'December;']\n",
      "['And', 'each', 'separate', 'dying', 'ember', 'wrought', 'its', 'ghost', 'upon', 'the', 'floor.']\n",
      "['Eagerly', 'I', 'wished', 'the', 'morrow;—vainly', 'I', 'had', 'sought', 'to', 'borrow']\n",
      "['From', 'my', 'books', 'surcease', 'of', 'sorrow—sorrow', 'for', 'the', 'lost', 'Lenore—']\n",
      "['For', 'the', 'rare', 'and', 'radiant', 'maiden', 'whom', 'the', 'angels', 'name', 'Lenore—']\n",
      "['Nameless', 'here', 'for', 'evermore.']\n",
      "[]\n",
      "['And', 'the', 'silken,', 'sad,', 'uncertain', 'rustling', 'of', 'each', 'purple', 'curtain']\n",
      "['Thrilled', 'me—filled', 'me', 'with', 'fantastic', 'terrors', 'never', 'felt', 'before;']\n",
      "['So', 'that', 'now,', 'to', 'still', 'the', 'beating', 'of', 'my', 'heart,', 'I', 'stood', 'repeating']\n",
      "['“’Tis', 'some', 'visitor', 'entreating', 'entrance', 'at', 'my', 'chamber', 'door—']\n",
      "['Some', 'late', 'visitor', 'entreating', 'entrance', 'at', 'my', 'chamber', 'door;—']\n",
      "['This', 'it', 'is', 'and', 'nothing', 'more.”']\n",
      "[]\n",
      "['Presently', 'my', 'soul', 'grew', 'stronger;', 'hesitating', 'then', 'no', 'longer,']\n",
      "['“Sir,”', 'said', 'I,', '“or', 'Madam,', 'truly', 'your', 'forgiveness', 'I', 'implore;']\n",
      "['But', 'the', 'fact', 'is', 'I', 'was', 'napping,', 'and', 'so', 'gently', 'you', 'came', 'rapping,']\n",
      "['And', 'so', 'faintly', 'you', 'came', 'tapping,', 'tapping', 'at', 'my', 'chamber', 'door,']\n",
      "['That', 'I', 'scarce', 'was', 'sure', 'I', 'heard', 'you”—here', 'I', 'opened', 'wide', 'the', 'door;—']\n",
      "['Darkness', 'there', 'and', 'nothing', 'more.']\n",
      "[]\n",
      "['Deep', 'into', 'that', 'darkness', 'peering,', 'long', 'I', 'stood', 'there', 'wondering,', 'fearing,']\n",
      "['Doubting,', 'dreaming', 'dreams', 'no', 'mortal', 'ever', 'dared', 'to', 'dream', 'before;']\n",
      "['But', 'the', 'silence', 'was', 'unbroken,', 'and', 'the', 'stillness', 'gave', 'no', 'token,']\n",
      "['And', 'the', 'only', 'word', 'there', 'spoken', 'was', 'the', 'whispered', 'word,', '“Lenore?”']\n",
      "['This', 'I', 'whispered,', 'and', 'an', 'echo', 'murmured', 'back', 'the', 'word,', '“Lenore!”—']\n",
      "['Merely', 'this', 'and', 'nothing', 'more.']\n",
      "[]\n",
      "['Back', 'into', 'the', 'chamber', 'turning,', 'all', 'my', 'soul', 'within', 'me', 'burning,']\n",
      "['Soon', 'again', 'I', 'heard', 'a', 'tapping', 'somewhat', 'louder', 'than', 'before.']\n",
      "['“Surely,”', 'said', 'I,', '“surely', 'that', 'is', 'something', 'at', 'my', 'window', 'lattice;']\n",
      "['Let', 'me', 'see,', 'then,', 'what', 'thereat', 'is,', 'and', 'this', 'mystery', 'explore—']\n",
      "['Let', 'my', 'heart', 'be', 'still', 'a', 'moment', 'and', 'this', 'mystery', 'explore;—']\n",
      "['’Tis', 'the', 'wind', 'and', 'nothing', 'more!”']\n",
      "[]\n",
      "['Open', 'here', 'I', 'flung', 'the', 'shutter,', 'when,', 'with', 'many', 'a', 'flirt', 'and', 'flutter,']\n",
      "['In', 'there', 'stepped', 'a', 'stately', 'Raven', 'of', 'the', 'saintly', 'days', 'of', 'yore;']\n",
      "['Not', 'the', 'least', 'obeisance', 'made', 'he;', 'not', 'a', 'minute', 'stopped', 'or', 'stayed', 'he;']\n",
      "['But,', 'with', 'mien', 'of', 'lord', 'or', 'lady,', 'perched', 'above', 'my', 'chamber', 'door—']\n",
      "['Perched', 'upon', 'a', 'bust', 'of', 'Pallas', 'just', 'above', 'my', 'chamber', 'door—']\n",
      "['Perched,', 'and', 'sat,', 'and', 'nothing', 'more.']\n",
      "[]\n",
      "['Then', 'this', 'ebony', 'bird', 'beguiling', 'my', 'sad', 'fancy', 'into', 'smiling,']\n",
      "['By', 'the', 'grave', 'and', 'stern', 'decorum', 'of', 'the', 'countenance', 'it', 'wore,']\n",
      "['“Though', 'thy', 'crest', 'be', 'shorn', 'and', 'shaven,', 'thou,”', 'I', 'said,', '“art', 'sure', 'no', 'craven,']\n",
      "['Ghastly', 'grim', 'and', 'ancient', 'Raven', 'wandering', 'from', 'the', 'Nightly', 'shore—']\n",
      "['Tell', 'me', 'what', 'thy', 'lordly', 'name', 'is', 'on', 'the', 'Night’s', 'Plutonian', 'shore!”']\n",
      "['Quoth', 'the', 'Raven', '“Nevermore.”']\n",
      "[]\n",
      "['Much', 'I', 'marvelled', 'this', 'ungainly', 'fowl', 'to', 'hear', 'discourse', 'so', 'plainly,']\n",
      "['Though', 'its', 'answer', 'little', 'meaning—little', 'relevancy', 'bore;']\n",
      "['For', 'we', 'cannot', 'help', 'agreeing', 'that', 'no', 'living', 'human', 'being']\n",
      "['Ever', 'yet', 'was', 'blessed', 'with', 'seeing', 'bird', 'above', 'his', 'chamber', 'door—']\n",
      "['Bird', 'or', 'beast', 'upon', 'the', 'sculptured', 'bust', 'above', 'his', 'chamber', 'door,']\n",
      "['With', 'such', 'name', 'as', '“Nevermore.”']\n",
      "[]\n",
      "['But', 'the', 'Raven,', 'sitting', 'lonely', 'on', 'the', 'placid', 'bust,', 'spoke', 'only']\n",
      "['That', 'one', 'word,', 'as', 'if', 'his', 'soul', 'in', 'that', 'one', 'word', 'he', 'did', 'outpour.']\n",
      "['Nothing', 'farther', 'then', 'he', 'uttered—not', 'a', 'feather', 'then', 'he', 'fluttered—']\n",
      "['Till', 'I', 'scarcely', 'more', 'than', 'muttered', '“Other', 'friends', 'have', 'flown', 'before—']\n",
      "['On', 'the', 'morrow', 'he', 'will', 'leave', 'me,', 'as', 'my', 'Hopes', 'have', 'flown', 'before.”']\n",
      "['Then', 'the', 'bird', 'said', '“Nevermore.”']\n",
      "[]\n",
      "['Startled', 'at', 'the', 'stillness', 'broken', 'by', 'reply', 'so', 'aptly', 'spoken,']\n",
      "['“Doubtless,”', 'said', 'I,', '“what', 'it', 'utters', 'is', 'its', 'only', 'stock', 'and', 'store']\n",
      "['Caught', 'from', 'some', 'unhappy', 'master', 'whom', 'unmerciful', 'Disaster']\n",
      "['Followed', 'fast', 'and', 'followed', 'faster', 'till', 'his', 'songs', 'one', 'burden', 'bore—']\n",
      "['Till', 'the', 'dirges', 'of', 'his', 'Hope', 'that', 'melancholy', 'burden', 'bore']\n",
      "['Of', '‘Never—nevermore’.”']\n",
      "[]\n",
      "['But', 'the', 'Raven', 'still', 'beguiling', 'all', 'my', 'fancy', 'into', 'smiling,']\n",
      "['Straight', 'I', 'wheeled', 'a', 'cushioned', 'seat', 'in', 'front', 'of', 'bird,', 'and', 'bust', 'and', 'door;']\n",
      "['Then,', 'upon', 'the', 'velvet', 'sinking,', 'I', 'betook', 'myself', 'to', 'linking']\n",
      "['Fancy', 'unto', 'fancy,', 'thinking', 'what', 'this', 'ominous', 'bird', 'of', 'yore—']\n",
      "['What', 'this', 'grim,', 'ungainly,', 'ghastly,', 'gaunt,', 'and', 'ominous', 'bird', 'of', 'yore']\n",
      "['Meant', 'in', 'croaking', '“Nevermore.”']\n",
      "[]\n",
      "['This', 'I', 'sat', 'engaged', 'in', 'guessing,', 'but', 'no', 'syllable', 'expressing']\n",
      "['To', 'the', 'fowl', 'whose', 'fiery', 'eyes', 'now', 'burned', 'into', 'my', 'bosom’s', 'core;']\n",
      "['This', 'and', 'more', 'I', 'sat', 'divining,', 'with', 'my', 'head', 'at', 'ease', 'reclining']\n",
      "['On', 'the', 'cushion’s', 'velvet', 'lining', 'that', 'the', 'lamp-light', 'gloated', 'o’er,']\n",
      "['But', 'whose', 'velvet-violet', 'lining', 'with', 'the', 'lamp-light', 'gloating', 'o’er,']\n",
      "['She', 'shall', 'press,', 'ah,', 'nevermore!']\n",
      "[]\n",
      "['Then,', 'methought,', 'the', 'air', 'grew', 'denser,', 'perfumed', 'from', 'an', 'unseen', 'censer']\n",
      "['Swung', 'by', 'Seraphim', 'whose', 'foot-falls', 'tinkled', 'on', 'the', 'tufted', 'floor.']\n",
      "['“Wretch,”', 'I', 'cried,', '“thy', 'God', 'hath', 'lent', 'thee—by', 'these', 'angels', 'he', 'hath', 'sent', 'thee']\n",
      "['Respite—respite', 'and', 'nepenthe', 'from', 'thy', 'memories', 'of', 'Lenore;']\n",
      "['Quaff,', 'oh', 'quaff', 'this', 'kind', 'nepenthe', 'and', 'forget', 'this', 'lost', 'Lenore!”']\n",
      "['Quoth', 'the', 'Raven', '“Nevermore.”']\n",
      "[]\n",
      "['“Prophet!”', 'said', 'I,', '“thing', 'of', 'evil!—prophet', 'still,', 'if', 'bird', 'or', 'devil!—']\n",
      "['Whether', 'Tempter', 'sent,', 'or', 'whether', 'tempest', 'tossed', 'thee', 'here', 'ashore,']\n",
      "['Desolate', 'yet', 'all', 'undaunted,', 'on', 'this', 'desert', 'land', 'enchanted—']\n",
      "['On', 'this', 'home', 'by', 'Horror', 'haunted—tell', 'me', 'truly,', 'I', 'implore—']\n",
      "['Is', 'there—is', 'there', 'balm', 'in', 'Gilead?—tell', 'me—tell', 'me,', 'I', 'implore!”']\n",
      "['Quoth', 'the', 'Raven', '“Nevermore.”']\n",
      "[]\n",
      "['“Prophet!”', 'said', 'I,', '“thing', 'of', 'evil!—prophet', 'still,', 'if', 'bird', 'or', 'devil!']\n",
      "['By', 'that', 'Heaven', 'that', 'bends', 'above', 'us—by', 'that', 'God', 'we', 'both', 'adore—']\n",
      "['Tell', 'this', 'soul', 'with', 'sorrow', 'laden', 'if,', 'within', 'the', 'distant', 'Aidenn,']\n",
      "['It', 'shall', 'clasp', 'a', 'sainted', 'maiden', 'whom', 'the', 'angels', 'name', 'Lenore—']\n",
      "['Clasp', 'a', 'rare', 'and', 'radiant', 'maiden', 'whom', 'the', 'angels', 'name', 'Lenore.”']\n",
      "['Quoth', 'the', 'Raven', '“Nevermore.”']\n",
      "[]\n",
      "['“Be', 'that', 'word', 'our', 'sign', 'of', 'parting,', 'bird', 'or', 'fiend!”', 'I', 'shrieked,', 'upstarting—']\n",
      "['“Get', 'thee', 'back', 'into', 'the', 'tempest', 'and', 'the', 'Night’s', 'Plutonian', 'shore!']\n",
      "['Leave', 'no', 'black', 'plume', 'as', 'a', 'token', 'of', 'that', 'lie', 'thy', 'soul', 'hath', 'spoken!']\n",
      "['Leave', 'my', 'loneliness', 'unbroken!—quit', 'the', 'bust', 'above', 'my', 'door!']\n",
      "['Take', 'thy', 'beak', 'from', 'out', 'my', 'heart,', 'and', 'take', 'thy', 'form', 'from', 'off', 'my', 'door!”']\n",
      "['Quoth', 'the', 'Raven', '“Nevermore.”']\n",
      "[]\n",
      "['And', 'the', 'Raven,', 'never', 'flitting,', 'still', 'is', 'sitting,', 'still', 'is', 'sitting']\n",
      "['On', 'the', 'pallid', 'bust', 'of', 'Pallas', 'just', 'above', 'my', 'chamber', 'door;']\n",
      "['And', 'his', 'eyes', 'have', 'all', 'the', 'seeming', 'of', 'a', 'demon’s', 'that', 'is', 'dreaming,']\n",
      "['And', 'the', 'lamp-light', 'o’er', 'him', 'streaming', 'throws', 'his', 'shadow', 'on', 'the', 'floor;']\n",
      "['And', 'my', 'soul', 'from', 'out', 'that', 'shadow', 'that', 'lies', 'floating', 'on', 'the', 'floor']\n",
      "['Shall', 'be', 'lifted—nevermore!']\n"
     ]
    }
   ],
   "source": [
    "infile = open('raven.txt', 'r') # makes our fileio object\n",
    "\n",
    "for line in infile: # loop over the lines in the file io object\n",
    "    # we've got our lines coming in, so we can start messing about with the content\n",
    "    words = line.split()\n",
    "    print(words)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Looks good!  We've got our lines being split into words.  Let's try to get the first word out of each."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 13,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Once\n",
      "Over\n",
      "While\n",
      "As\n",
      "“’Tis\n",
      "Only\n"
     ]
    },
    {
     "ename": "IndexError",
     "evalue": "list index out of range",
     "output_type": "error",
     "traceback": [
      "\u001b[0;31m--------------------------------------------------\u001b[0m",
      "\u001b[0;31mIndexError\u001b[0m       Traceback (most recent call last)",
      "\u001b[0;32m<ipython-input-13-3afdac67b397>\u001b[0m in \u001b[0;36m<module>\u001b[0;34m()\u001b[0m\n\u001b[1;32m      3\u001b[0m \u001b[0;32mfor\u001b[0m \u001b[0mline\u001b[0m \u001b[0;32min\u001b[0m \u001b[0minfile\u001b[0m\u001b[0;34m:\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m      4\u001b[0m     \u001b[0mwords\u001b[0m \u001b[0;34m=\u001b[0m \u001b[0mline\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0msplit\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0;32m----> 5\u001b[0;31m     \u001b[0mprint\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mwords\u001b[0m\u001b[0;34m[\u001b[0m\u001b[0;36m0\u001b[0m\u001b[0;34m]\u001b[0m\u001b[0;34m)\u001b[0m \u001b[0;31m# let's add our position lookup\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0m",
      "\u001b[0;31mIndexError\u001b[0m: list index out of range"
     ]
    }
   ],
   "source": [
    "infile = open('raven.txt', 'r')\n",
    "\n",
    "for line in infile:\n",
    "    words = line.split()\n",
    "    print(words[0]) # let's add our position lookup"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "That's surprising, but we can look at our lists coming out of the lines.  When the lines with just the newline are brought in and split, it returns an empty list.  So we can't access the first element of an empty list. \n",
    "\n",
    "We don't yet have the trick to do a boolean check of the length, so we'll need to back up and think about how we can prepare our file.\n",
    "\n",
    "The problem here are the newlines, which we can mess with more directly if we read everything in as a string.  Which would be `.read()`.\n",
    "\n",
    "So we know that we will want the lines out, so splitting on `\\n`.  Our problem is that we've got these `\\n\\n` pairs that cause an empty string to be returned.  We can use `.replace()` to change these `\\n\\n` into `\\n`.  We still want to keep a newline in there so we can split it up the way we want."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 18,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "['    Once upon a midnight dreary, while I pondered, weak and weary,', 'Over many a quaint and curious volume of forgotten lore—', '    While I nodded, nearly napping, suddenly there came a tapping,', 'As of some one gently rapping, rapping at my chamber door.', '“’Tis some visitor,” I muttered, “tapping at my chamber door—', '            Only this and nothing more.”', '    Ah, distinctly I remember it was in the bleak December;', 'And each separate dying ember wrought its ghost upon the floor.', '    Eagerly I wished the morrow;—vainly I had sought to borrow', '    From my books surcease of sorrow—sorrow for the lost Lenore—', 'For the rare and radiant maiden whom the angels name Lenore—', '            Nameless here for evermore.', '    And the silken, sad, uncertain rustling of each purple curtain', 'Thrilled me—filled me with fantastic terrors never felt before;', '    So that now, to still the beating of my heart, I stood repeating', '    “’Tis some visitor entreating entrance at my chamber door—', 'Some late visitor entreating entrance at my chamber door;—', '            This it is and nothing more.”', '    Presently my soul grew stronger; hesitating then no longer,', '“Sir,” said I, “or Madam, truly your forgiveness I implore;', '    But the fact is I was napping, and so gently you came rapping,', '    And so faintly you came tapping, tapping at my chamber door,', 'That I scarce was sure I heard you”—here I opened wide the door;—', '            Darkness there and nothing more.', '    Deep into that darkness peering, long I stood there wondering, fearing,', 'Doubting, dreaming dreams no mortal ever dared to dream before;', '    But the silence was unbroken, and the stillness gave no token,', '    And the only word there spoken was the whispered word, “Lenore?”', 'This I whispered, and an echo murmured back the word, “Lenore!”—', '            Merely this and nothing more.', '    Back into the chamber turning, all my soul within me burning,', 'Soon again I heard a tapping somewhat louder than before.', '    “Surely,” said I, “surely that is something at my window lattice;', '      Let me see, then, what thereat is, and this mystery explore—', 'Let my heart be still a moment and this mystery explore;—', '            ’Tis the wind and nothing more!”', '    Open here I flung the shutter, when, with many a flirt and flutter,', 'In there stepped a stately Raven of the saintly days of yore;', '    Not the least obeisance made he; not a minute stopped or stayed he;', '    But, with mien of lord or lady, perched above my chamber door—', 'Perched upon a bust of Pallas just above my chamber door—', '            Perched, and sat, and nothing more.', '    Then this ebony bird beguiling my sad fancy into smiling,', 'By the grave and stern decorum of the countenance it wore,', '“Though thy crest be shorn and shaven, thou,” I said, “art sure no craven,', 'Ghastly grim and ancient Raven wandering from the Nightly shore—', 'Tell me what thy lordly name is on the Night’s Plutonian shore!”', '            Quoth the Raven “Nevermore.”', '    Much I marvelled this ungainly fowl to hear discourse so plainly,', 'Though its answer little meaning—little relevancy bore;', '    For we cannot help agreeing that no living human being', '    Ever yet was blessed with seeing bird above his chamber door—', 'Bird or beast upon the sculptured bust above his chamber door,', '            With such name as “Nevermore.”', '    But the Raven, sitting lonely on the placid bust, spoke only', 'That one word, as if his soul in that one word he did outpour.', '    Nothing farther then he uttered—not a feather then he fluttered—', '    Till I scarcely more than muttered “Other friends have flown before—', 'On the morrow he will leave me, as my Hopes have flown before.”', '            Then the bird said “Nevermore.”', '    Startled at the stillness broken by reply so aptly spoken,', '“Doubtless,” said I, “what it utters is its only stock and store', '    Caught from some unhappy master whom unmerciful Disaster', '    Followed fast and followed faster till his songs one burden bore—', 'Till the dirges of his Hope that melancholy burden bore', '            Of ‘Never—nevermore’.”', '    But the Raven still beguiling all my fancy into smiling,', 'Straight I wheeled a cushioned seat in front of bird, and bust and door;', '    Then, upon the velvet sinking, I betook myself to linking', '    Fancy unto fancy, thinking what this ominous bird of yore—', 'What this grim, ungainly, ghastly, gaunt, and ominous bird of yore', '            Meant in croaking “Nevermore.”', '    This I sat engaged in guessing, but no syllable expressing', 'To the fowl whose fiery eyes now burned into my bosom’s core;', '    This and more I sat divining, with my head at ease reclining', '    On the cushion’s velvet lining that the lamp-light gloated o’er,', 'But whose velvet-violet lining with the lamp-light gloating o’er,', '            She shall press, ah, nevermore!', '    Then, methought, the air grew denser, perfumed from an unseen censer', 'Swung by Seraphim whose foot-falls tinkled on the tufted floor.', '    “Wretch,” I cried, “thy God hath lent thee—by these angels he hath sent thee', '    Respite—respite and nepenthe from thy memories of Lenore;', 'Quaff, oh quaff this kind nepenthe and forget this lost Lenore!”', '            Quoth the Raven “Nevermore.”', '    “Prophet!” said I, “thing of evil!—prophet still, if bird or devil!—', 'Whether Tempter sent, or whether tempest tossed thee here ashore,', '    Desolate yet all undaunted, on this desert land enchanted—', '    On this home by Horror haunted—tell me truly, I implore—', 'Is there—is there balm in Gilead?—tell me—tell me, I implore!”', '            Quoth the Raven “Nevermore.”', '    “Prophet!” said I, “thing of evil!—prophet still, if bird or devil!', 'By that Heaven that bends above us—by that God we both adore—', '    Tell this soul with sorrow laden if, within the distant Aidenn,', '    It shall clasp a sainted maiden whom the angels name Lenore—', 'Clasp a rare and radiant maiden whom the angels name Lenore.”', '            Quoth the Raven “Nevermore.”', '    “Be that word our sign of parting, bird or fiend!” I shrieked, upstarting—', '“Get thee back into the tempest and the Night’s Plutonian shore!', '    Leave no black plume as a token of that lie thy soul hath spoken!', '    Leave my loneliness unbroken!—quit the bust above my door!', 'Take thy beak from out my heart, and take thy form from off my door!”', '            Quoth the Raven “Nevermore.”', '    And the Raven, never flitting, still is sitting, still is sitting', 'On the pallid bust of Pallas just above my chamber door;', '    And his eyes have all the seeming of a demon’s that is dreaming,', '    And the lamp-light o’er him streaming throws his shadow on the floor;', 'And my soul from out that shadow that lies floating on the floor', '            Shall be lifted—nevermore!']\n"
     ]
    }
   ],
   "source": [
    "infile = open('raven.txt', 'r') # makes our fileio object\n",
    "\n",
    "text = infile.read()\n",
    "\n",
    "infile.close()\n",
    "\n",
    "corrected_text = text.replace('\\n\\n', '\\n')\n",
    "lines = corrected_text.split('\\n')\n",
    "print(lines)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Now we can loop through our lines and try splitting them up again."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 20,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "['Once', 'upon', 'a', 'midnight', 'dreary,', 'while', 'I', 'pondered,', 'weak', 'and', 'weary,']\n",
      "['Over', 'many', 'a', 'quaint', 'and', 'curious', 'volume', 'of', 'forgotten', 'lore—']\n",
      "['While', 'I', 'nodded,', 'nearly', 'napping,', 'suddenly', 'there', 'came', 'a', 'tapping,']\n",
      "['As', 'of', 'some', 'one', 'gently', 'rapping,', 'rapping', 'at', 'my', 'chamber', 'door.']\n",
      "['“’Tis', 'some', 'visitor,”', 'I', 'muttered,', '“tapping', 'at', 'my', 'chamber', 'door—']\n",
      "['Only', 'this', 'and', 'nothing', 'more.”']\n",
      "['Ah,', 'distinctly', 'I', 'remember', 'it', 'was', 'in', 'the', 'bleak', 'December;']\n",
      "['And', 'each', 'separate', 'dying', 'ember', 'wrought', 'its', 'ghost', 'upon', 'the', 'floor.']\n",
      "['Eagerly', 'I', 'wished', 'the', 'morrow;—vainly', 'I', 'had', 'sought', 'to', 'borrow']\n",
      "['From', 'my', 'books', 'surcease', 'of', 'sorrow—sorrow', 'for', 'the', 'lost', 'Lenore—']\n",
      "['For', 'the', 'rare', 'and', 'radiant', 'maiden', 'whom', 'the', 'angels', 'name', 'Lenore—']\n",
      "['Nameless', 'here', 'for', 'evermore.']\n",
      "['And', 'the', 'silken,', 'sad,', 'uncertain', 'rustling', 'of', 'each', 'purple', 'curtain']\n",
      "['Thrilled', 'me—filled', 'me', 'with', 'fantastic', 'terrors', 'never', 'felt', 'before;']\n",
      "['So', 'that', 'now,', 'to', 'still', 'the', 'beating', 'of', 'my', 'heart,', 'I', 'stood', 'repeating']\n",
      "['“’Tis', 'some', 'visitor', 'entreating', 'entrance', 'at', 'my', 'chamber', 'door—']\n",
      "['Some', 'late', 'visitor', 'entreating', 'entrance', 'at', 'my', 'chamber', 'door;—']\n",
      "['This', 'it', 'is', 'and', 'nothing', 'more.”']\n",
      "['Presently', 'my', 'soul', 'grew', 'stronger;', 'hesitating', 'then', 'no', 'longer,']\n",
      "['“Sir,”', 'said', 'I,', '“or', 'Madam,', 'truly', 'your', 'forgiveness', 'I', 'implore;']\n",
      "['But', 'the', 'fact', 'is', 'I', 'was', 'napping,', 'and', 'so', 'gently', 'you', 'came', 'rapping,']\n",
      "['And', 'so', 'faintly', 'you', 'came', 'tapping,', 'tapping', 'at', 'my', 'chamber', 'door,']\n",
      "['That', 'I', 'scarce', 'was', 'sure', 'I', 'heard', 'you”—here', 'I', 'opened', 'wide', 'the', 'door;—']\n",
      "['Darkness', 'there', 'and', 'nothing', 'more.']\n",
      "['Deep', 'into', 'that', 'darkness', 'peering,', 'long', 'I', 'stood', 'there', 'wondering,', 'fearing,']\n",
      "['Doubting,', 'dreaming', 'dreams', 'no', 'mortal', 'ever', 'dared', 'to', 'dream', 'before;']\n",
      "['But', 'the', 'silence', 'was', 'unbroken,', 'and', 'the', 'stillness', 'gave', 'no', 'token,']\n",
      "['And', 'the', 'only', 'word', 'there', 'spoken', 'was', 'the', 'whispered', 'word,', '“Lenore?”']\n",
      "['This', 'I', 'whispered,', 'and', 'an', 'echo', 'murmured', 'back', 'the', 'word,', '“Lenore!”—']\n",
      "['Merely', 'this', 'and', 'nothing', 'more.']\n",
      "['Back', 'into', 'the', 'chamber', 'turning,', 'all', 'my', 'soul', 'within', 'me', 'burning,']\n",
      "['Soon', 'again', 'I', 'heard', 'a', 'tapping', 'somewhat', 'louder', 'than', 'before.']\n",
      "['“Surely,”', 'said', 'I,', '“surely', 'that', 'is', 'something', 'at', 'my', 'window', 'lattice;']\n",
      "['Let', 'me', 'see,', 'then,', 'what', 'thereat', 'is,', 'and', 'this', 'mystery', 'explore—']\n",
      "['Let', 'my', 'heart', 'be', 'still', 'a', 'moment', 'and', 'this', 'mystery', 'explore;—']\n",
      "['’Tis', 'the', 'wind', 'and', 'nothing', 'more!”']\n",
      "['Open', 'here', 'I', 'flung', 'the', 'shutter,', 'when,', 'with', 'many', 'a', 'flirt', 'and', 'flutter,']\n",
      "['In', 'there', 'stepped', 'a', 'stately', 'Raven', 'of', 'the', 'saintly', 'days', 'of', 'yore;']\n",
      "['Not', 'the', 'least', 'obeisance', 'made', 'he;', 'not', 'a', 'minute', 'stopped', 'or', 'stayed', 'he;']\n",
      "['But,', 'with', 'mien', 'of', 'lord', 'or', 'lady,', 'perched', 'above', 'my', 'chamber', 'door—']\n",
      "['Perched', 'upon', 'a', 'bust', 'of', 'Pallas', 'just', 'above', 'my', 'chamber', 'door—']\n",
      "['Perched,', 'and', 'sat,', 'and', 'nothing', 'more.']\n",
      "['Then', 'this', 'ebony', 'bird', 'beguiling', 'my', 'sad', 'fancy', 'into', 'smiling,']\n",
      "['By', 'the', 'grave', 'and', 'stern', 'decorum', 'of', 'the', 'countenance', 'it', 'wore,']\n",
      "['“Though', 'thy', 'crest', 'be', 'shorn', 'and', 'shaven,', 'thou,”', 'I', 'said,', '“art', 'sure', 'no', 'craven,']\n",
      "['Ghastly', 'grim', 'and', 'ancient', 'Raven', 'wandering', 'from', 'the', 'Nightly', 'shore—']\n",
      "['Tell', 'me', 'what', 'thy', 'lordly', 'name', 'is', 'on', 'the', 'Night’s', 'Plutonian', 'shore!”']\n",
      "['Quoth', 'the', 'Raven', '“Nevermore.”']\n",
      "['Much', 'I', 'marvelled', 'this', 'ungainly', 'fowl', 'to', 'hear', 'discourse', 'so', 'plainly,']\n",
      "['Though', 'its', 'answer', 'little', 'meaning—little', 'relevancy', 'bore;']\n",
      "['For', 'we', 'cannot', 'help', 'agreeing', 'that', 'no', 'living', 'human', 'being']\n",
      "['Ever', 'yet', 'was', 'blessed', 'with', 'seeing', 'bird', 'above', 'his', 'chamber', 'door—']\n",
      "['Bird', 'or', 'beast', 'upon', 'the', 'sculptured', 'bust', 'above', 'his', 'chamber', 'door,']\n",
      "['With', 'such', 'name', 'as', '“Nevermore.”']\n",
      "['But', 'the', 'Raven,', 'sitting', 'lonely', 'on', 'the', 'placid', 'bust,', 'spoke', 'only']\n",
      "['That', 'one', 'word,', 'as', 'if', 'his', 'soul', 'in', 'that', 'one', 'word', 'he', 'did', 'outpour.']\n",
      "['Nothing', 'farther', 'then', 'he', 'uttered—not', 'a', 'feather', 'then', 'he', 'fluttered—']\n",
      "['Till', 'I', 'scarcely', 'more', 'than', 'muttered', '“Other', 'friends', 'have', 'flown', 'before—']\n",
      "['On', 'the', 'morrow', 'he', 'will', 'leave', 'me,', 'as', 'my', 'Hopes', 'have', 'flown', 'before.”']\n",
      "['Then', 'the', 'bird', 'said', '“Nevermore.”']\n",
      "['Startled', 'at', 'the', 'stillness', 'broken', 'by', 'reply', 'so', 'aptly', 'spoken,']\n",
      "['“Doubtless,”', 'said', 'I,', '“what', 'it', 'utters', 'is', 'its', 'only', 'stock', 'and', 'store']\n",
      "['Caught', 'from', 'some', 'unhappy', 'master', 'whom', 'unmerciful', 'Disaster']\n",
      "['Followed', 'fast', 'and', 'followed', 'faster', 'till', 'his', 'songs', 'one', 'burden', 'bore—']\n",
      "['Till', 'the', 'dirges', 'of', 'his', 'Hope', 'that', 'melancholy', 'burden', 'bore']\n",
      "['Of', '‘Never—nevermore’.”']\n",
      "['But', 'the', 'Raven', 'still', 'beguiling', 'all', 'my', 'fancy', 'into', 'smiling,']\n",
      "['Straight', 'I', 'wheeled', 'a', 'cushioned', 'seat', 'in', 'front', 'of', 'bird,', 'and', 'bust', 'and', 'door;']\n",
      "['Then,', 'upon', 'the', 'velvet', 'sinking,', 'I', 'betook', 'myself', 'to', 'linking']\n",
      "['Fancy', 'unto', 'fancy,', 'thinking', 'what', 'this', 'ominous', 'bird', 'of', 'yore—']\n",
      "['What', 'this', 'grim,', 'ungainly,', 'ghastly,', 'gaunt,', 'and', 'ominous', 'bird', 'of', 'yore']\n",
      "['Meant', 'in', 'croaking', '“Nevermore.”']\n",
      "['This', 'I', 'sat', 'engaged', 'in', 'guessing,', 'but', 'no', 'syllable', 'expressing']\n",
      "['To', 'the', 'fowl', 'whose', 'fiery', 'eyes', 'now', 'burned', 'into', 'my', 'bosom’s', 'core;']\n",
      "['This', 'and', 'more', 'I', 'sat', 'divining,', 'with', 'my', 'head', 'at', 'ease', 'reclining']\n",
      "['On', 'the', 'cushion’s', 'velvet', 'lining', 'that', 'the', 'lamp-light', 'gloated', 'o’er,']\n",
      "['But', 'whose', 'velvet-violet', 'lining', 'with', 'the', 'lamp-light', 'gloating', 'o’er,']\n",
      "['She', 'shall', 'press,', 'ah,', 'nevermore!']\n",
      "['Then,', 'methought,', 'the', 'air', 'grew', 'denser,', 'perfumed', 'from', 'an', 'unseen', 'censer']\n",
      "['Swung', 'by', 'Seraphim', 'whose', 'foot-falls', 'tinkled', 'on', 'the', 'tufted', 'floor.']\n",
      "['“Wretch,”', 'I', 'cried,', '“thy', 'God', 'hath', 'lent', 'thee—by', 'these', 'angels', 'he', 'hath', 'sent', 'thee']\n",
      "['Respite—respite', 'and', 'nepenthe', 'from', 'thy', 'memories', 'of', 'Lenore;']\n",
      "['Quaff,', 'oh', 'quaff', 'this', 'kind', 'nepenthe', 'and', 'forget', 'this', 'lost', 'Lenore!”']\n",
      "['Quoth', 'the', 'Raven', '“Nevermore.”']\n",
      "['“Prophet!”', 'said', 'I,', '“thing', 'of', 'evil!—prophet', 'still,', 'if', 'bird', 'or', 'devil!—']\n",
      "['Whether', 'Tempter', 'sent,', 'or', 'whether', 'tempest', 'tossed', 'thee', 'here', 'ashore,']\n",
      "['Desolate', 'yet', 'all', 'undaunted,', 'on', 'this', 'desert', 'land', 'enchanted—']\n",
      "['On', 'this', 'home', 'by', 'Horror', 'haunted—tell', 'me', 'truly,', 'I', 'implore—']\n",
      "['Is', 'there—is', 'there', 'balm', 'in', 'Gilead?—tell', 'me—tell', 'me,', 'I', 'implore!”']\n",
      "['Quoth', 'the', 'Raven', '“Nevermore.”']\n",
      "['“Prophet!”', 'said', 'I,', '“thing', 'of', 'evil!—prophet', 'still,', 'if', 'bird', 'or', 'devil!']\n",
      "['By', 'that', 'Heaven', 'that', 'bends', 'above', 'us—by', 'that', 'God', 'we', 'both', 'adore—']\n",
      "['Tell', 'this', 'soul', 'with', 'sorrow', 'laden', 'if,', 'within', 'the', 'distant', 'Aidenn,']\n",
      "['It', 'shall', 'clasp', 'a', 'sainted', 'maiden', 'whom', 'the', 'angels', 'name', 'Lenore—']\n",
      "['Clasp', 'a', 'rare', 'and', 'radiant', 'maiden', 'whom', 'the', 'angels', 'name', 'Lenore.”']\n",
      "['Quoth', 'the', 'Raven', '“Nevermore.”']\n",
      "['“Be', 'that', 'word', 'our', 'sign', 'of', 'parting,', 'bird', 'or', 'fiend!”', 'I', 'shrieked,', 'upstarting—']\n",
      "['“Get', 'thee', 'back', 'into', 'the', 'tempest', 'and', 'the', 'Night’s', 'Plutonian', 'shore!']\n",
      "['Leave', 'no', 'black', 'plume', 'as', 'a', 'token', 'of', 'that', 'lie', 'thy', 'soul', 'hath', 'spoken!']\n",
      "['Leave', 'my', 'loneliness', 'unbroken!—quit', 'the', 'bust', 'above', 'my', 'door!']\n",
      "['Take', 'thy', 'beak', 'from', 'out', 'my', 'heart,', 'and', 'take', 'thy', 'form', 'from', 'off', 'my', 'door!”']\n",
      "['Quoth', 'the', 'Raven', '“Nevermore.”']\n",
      "['And', 'the', 'Raven,', 'never', 'flitting,', 'still', 'is', 'sitting,', 'still', 'is', 'sitting']\n",
      "['On', 'the', 'pallid', 'bust', 'of', 'Pallas', 'just', 'above', 'my', 'chamber', 'door;']\n",
      "['And', 'his', 'eyes', 'have', 'all', 'the', 'seeming', 'of', 'a', 'demon’s', 'that', 'is', 'dreaming,']\n",
      "['And', 'the', 'lamp-light', 'o’er', 'him', 'streaming', 'throws', 'his', 'shadow', 'on', 'the', 'floor;']\n",
      "['And', 'my', 'soul', 'from', 'out', 'that', 'shadow', 'that', 'lies', 'floating', 'on', 'the', 'floor']\n",
      "['Shall', 'be', 'lifted—nevermore!']\n"
     ]
    }
   ],
   "source": [
    "infile = open('raven.txt', 'r') # makes our fileio object\n",
    "\n",
    "text = infile.read()\n",
    "\n",
    "infile.close()\n",
    "\n",
    "corrected_text = text.replace('\\n\\n', '\\n')\n",
    "lines = corrected_text.split('\\n')\n",
    "\n",
    "for line in lines:\n",
    "    print(line.split()) # looks better!!\n",
    "    "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 21,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Once\n",
      "Over\n",
      "While\n",
      "As\n",
      "“’Tis\n",
      "Only\n",
      "Ah,\n",
      "And\n",
      "Eagerly\n",
      "From\n",
      "For\n",
      "Nameless\n",
      "And\n",
      "Thrilled\n",
      "So\n",
      "“’Tis\n",
      "Some\n",
      "This\n",
      "Presently\n",
      "“Sir,”\n",
      "But\n",
      "And\n",
      "That\n",
      "Darkness\n",
      "Deep\n",
      "Doubting,\n",
      "But\n",
      "And\n",
      "This\n",
      "Merely\n",
      "Back\n",
      "Soon\n",
      "“Surely,”\n",
      "Let\n",
      "Let\n",
      "’Tis\n",
      "Open\n",
      "In\n",
      "Not\n",
      "But,\n",
      "Perched\n",
      "Perched,\n",
      "Then\n",
      "By\n",
      "“Though\n",
      "Ghastly\n",
      "Tell\n",
      "Quoth\n",
      "Much\n",
      "Though\n",
      "For\n",
      "Ever\n",
      "Bird\n",
      "With\n",
      "But\n",
      "That\n",
      "Nothing\n",
      "Till\n",
      "On\n",
      "Then\n",
      "Startled\n",
      "“Doubtless,”\n",
      "Caught\n",
      "Followed\n",
      "Till\n",
      "Of\n",
      "But\n",
      "Straight\n",
      "Then,\n",
      "Fancy\n",
      "What\n",
      "Meant\n",
      "This\n",
      "To\n",
      "This\n",
      "On\n",
      "But\n",
      "She\n",
      "Then,\n",
      "Swung\n",
      "“Wretch,”\n",
      "Respite—respite\n",
      "Quaff,\n",
      "Quoth\n",
      "“Prophet!”\n",
      "Whether\n",
      "Desolate\n",
      "On\n",
      "Is\n",
      "Quoth\n",
      "“Prophet!”\n",
      "By\n",
      "Tell\n",
      "It\n",
      "Clasp\n",
      "Quoth\n",
      "“Be\n",
      "“Get\n",
      "Leave\n",
      "Leave\n",
      "Take\n",
      "Quoth\n",
      "And\n",
      "On\n",
      "And\n",
      "And\n",
      "And\n",
      "Shall\n"
     ]
    }
   ],
   "source": [
    "infile = open('raven.txt', 'r') # makes our fileio object\n",
    "\n",
    "text = infile.read()\n",
    "\n",
    "infile.close()\n",
    "\n",
    "corrected_text = text.replace('\\n\\n', '\\n')\n",
    "lines = corrected_text.split('\\n')\n",
    "\n",
    "for line in lines:\n",
    "    words = line.split()\n",
    "    print(words[0])\n",
    "    "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Well, we've done the thing and you can tell your friend that they're full of crap!"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": []
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "collapsed": true
   },
   "source": [
    "# An example of applying a function to each line of a text file\n",
    "\n",
    "So let's say you have a file with one entry of data per line.  For example, in The Raven, let's say that one line of the text is one entry of data.  This is a relatively standard form of organizing data, because it is so easy to operate on data at the line level.\n",
    "\n",
    "Now let's say you have a function that you want to apply to that file, running each line of the file through that function.  You want to gather the results from this function, and as a start write out a file with those results.  This refers to the batch processing that is introduced within the Zelle book.\n",
    "\n",
    "For the sake of example, we're going to run each line through the `len()` function.  This will count the number of characters within each line of text.  You can replace `len` with any other function that accepts a single string value.  (like your acronym function!!)\n",
    "\n",
    "Let's build this up together, but first, come up with a battle plan.\n",
    "\n",
    "## step 1:  read through the file and access the text\n",
    "\n",
    "Read in the file and be able to access each line of text.  We can do this from a variety of approaches, but let's just get our plumbing in order.\n",
    "\n",
    "We're going to start with code that we've used before."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "    Once upon a midnight dreary, while I pondered, weak and weary,\n",
      "\n",
      "Over many a quaint and curious volume of forgotten lore—\n",
      "\n",
      "    While I nodded, nearly napping, suddenly there came a tapping,\n",
      "\n",
      "As of some one gently rapping, rapping at my chamber door.\n",
      "\n",
      "“’Tis some visitor,” I muttered, “tapping at my chamber door—\n",
      "\n",
      "            Only this and nothing more.”\n",
      "\n",
      "\n",
      "\n",
      "    Ah, distinctly I remember it was in the bleak December;\n",
      "\n",
      "And each separate dying ember wrought its ghost upon the floor.\n",
      "\n",
      "    Eagerly I wished the morrow;—vainly I had sought to borrow\n",
      "\n",
      "    From my books surcease of sorrow—sorrow for the lost Lenore—\n",
      "\n",
      "For the rare and radiant maiden whom the angels name Lenore—\n",
      "\n",
      "            Nameless here for evermore.\n",
      "\n",
      "\n",
      "\n",
      "    And the silken, sad, uncertain rustling of each purple curtain\n",
      "\n",
      "Thrilled me—filled me with fantastic terrors never felt before;\n",
      "\n",
      "    So that now, to still the beating of my heart, I stood repeating\n",
      "\n",
      "    “’Tis some visitor entreating entrance at my chamber door—\n",
      "\n",
      "Some late visitor entreating entrance at my chamber door;—\n",
      "\n",
      "            This it is and nothing more.”\n",
      "\n",
      "\n",
      "\n",
      "    Presently my soul grew stronger; hesitating then no longer,\n",
      "\n",
      "“Sir,” said I, “or Madam, truly your forgiveness I implore;\n",
      "\n",
      "    But the fact is I was napping, and so gently you came rapping,\n",
      "\n",
      "    And so faintly you came tapping, tapping at my chamber door,\n",
      "\n",
      "That I scarce was sure I heard you”—here I opened wide the door;—\n",
      "\n",
      "            Darkness there and nothing more.\n",
      "\n",
      "\n",
      "\n",
      "    Deep into that darkness peering, long I stood there wondering, fearing,\n",
      "\n",
      "Doubting, dreaming dreams no mortal ever dared to dream before;\n",
      "\n",
      "    But the silence was unbroken, and the stillness gave no token,\n",
      "\n",
      "    And the only word there spoken was the whispered word, “Lenore?”\n",
      "\n",
      "This I whispered, and an echo murmured back the word, “Lenore!”—\n",
      "\n",
      "            Merely this and nothing more.\n",
      "\n",
      "\n",
      "\n",
      "    Back into the chamber turning, all my soul within me burning,\n",
      "\n",
      "Soon again I heard a tapping somewhat louder than before.\n",
      "\n",
      "    “Surely,” said I, “surely that is something at my window lattice;\n",
      "\n",
      "      Let me see, then, what thereat is, and this mystery explore—\n",
      "\n",
      "Let my heart be still a moment and this mystery explore;—\n",
      "\n",
      "            ’Tis the wind and nothing more!”\n",
      "\n",
      "\n",
      "\n",
      "    Open here I flung the shutter, when, with many a flirt and flutter,\n",
      "\n",
      "In there stepped a stately Raven of the saintly days of yore;\n",
      "\n",
      "    Not the least obeisance made he; not a minute stopped or stayed he;\n",
      "\n",
      "    But, with mien of lord or lady, perched above my chamber door—\n",
      "\n",
      "Perched upon a bust of Pallas just above my chamber door—\n",
      "\n",
      "            Perched, and sat, and nothing more.\n",
      "\n",
      "\n",
      "\n",
      "    Then this ebony bird beguiling my sad fancy into smiling,\n",
      "\n",
      "By the grave and stern decorum of the countenance it wore,\n",
      "\n",
      "“Though thy crest be shorn and shaven, thou,” I said, “art sure no craven,\n",
      "\n",
      "Ghastly grim and ancient Raven wandering from the Nightly shore—\n",
      "\n",
      "Tell me what thy lordly name is on the Night’s Plutonian shore!”\n",
      "\n",
      "            Quoth the Raven “Nevermore.”\n",
      "\n",
      "\n",
      "\n",
      "    Much I marvelled this ungainly fowl to hear discourse so plainly,\n",
      "\n",
      "Though its answer little meaning—little relevancy bore;\n",
      "\n",
      "    For we cannot help agreeing that no living human being\n",
      "\n",
      "    Ever yet was blessed with seeing bird above his chamber door—\n",
      "\n",
      "Bird or beast upon the sculptured bust above his chamber door,\n",
      "\n",
      "            With such name as “Nevermore.”\n",
      "\n",
      "\n",
      "\n",
      "    But the Raven, sitting lonely on the placid bust, spoke only\n",
      "\n",
      "That one word, as if his soul in that one word he did outpour.\n",
      "\n",
      "    Nothing farther then he uttered—not a feather then he fluttered—\n",
      "\n",
      "    Till I scarcely more than muttered “Other friends have flown before—\n",
      "\n",
      "On the morrow he will leave me, as my Hopes have flown before.”\n",
      "\n",
      "            Then the bird said “Nevermore.”\n",
      "\n",
      "\n",
      "\n",
      "    Startled at the stillness broken by reply so aptly spoken,\n",
      "\n",
      "“Doubtless,” said I, “what it utters is its only stock and store\n",
      "\n",
      "    Caught from some unhappy master whom unmerciful Disaster\n",
      "\n",
      "    Followed fast and followed faster till his songs one burden bore—\n",
      "\n",
      "Till the dirges of his Hope that melancholy burden bore\n",
      "\n",
      "            Of ‘Never—nevermore’.”\n",
      "\n",
      "\n",
      "\n",
      "    But the Raven still beguiling all my fancy into smiling,\n",
      "\n",
      "Straight I wheeled a cushioned seat in front of bird, and bust and door;\n",
      "\n",
      "    Then, upon the velvet sinking, I betook myself to linking\n",
      "\n",
      "    Fancy unto fancy, thinking what this ominous bird of yore—\n",
      "\n",
      "What this grim, ungainly, ghastly, gaunt, and ominous bird of yore\n",
      "\n",
      "            Meant in croaking “Nevermore.”\n",
      "\n",
      "\n",
      "\n",
      "    This I sat engaged in guessing, but no syllable expressing\n",
      "\n",
      "To the fowl whose fiery eyes now burned into my bosom’s core;\n",
      "\n",
      "    This and more I sat divining, with my head at ease reclining\n",
      "\n",
      "    On the cushion’s velvet lining that the lamp-light gloated o’er,\n",
      "\n",
      "But whose velvet-violet lining with the lamp-light gloating o’er,\n",
      "\n",
      "            She shall press, ah, nevermore!\n",
      "\n",
      "\n",
      "\n",
      "    Then, methought, the air grew denser, perfumed from an unseen censer\n",
      "\n",
      "Swung by Seraphim whose foot-falls tinkled on the tufted floor.\n",
      "\n",
      "    “Wretch,” I cried, “thy God hath lent thee—by these angels he hath sent thee\n",
      "\n",
      "    Respite—respite and nepenthe from thy memories of Lenore;\n",
      "\n",
      "Quaff, oh quaff this kind nepenthe and forget this lost Lenore!”\n",
      "\n",
      "            Quoth the Raven “Nevermore.”\n",
      "\n",
      "\n",
      "\n",
      "    “Prophet!” said I, “thing of evil!—prophet still, if bird or devil!—\n",
      "\n",
      "Whether Tempter sent, or whether tempest tossed thee here ashore,\n",
      "\n",
      "    Desolate yet all undaunted, on this desert land enchanted—\n",
      "\n",
      "    On this home by Horror haunted—tell me truly, I implore—\n",
      "\n",
      "Is there—is there balm in Gilead?—tell me—tell me, I implore!”\n",
      "\n",
      "            Quoth the Raven “Nevermore.”\n",
      "\n",
      "\n",
      "\n",
      "    “Prophet!” said I, “thing of evil!—prophet still, if bird or devil!\n",
      "\n",
      "By that Heaven that bends above us—by that God we both adore—\n",
      "\n",
      "    Tell this soul with sorrow laden if, within the distant Aidenn,\n",
      "\n",
      "    It shall clasp a sainted maiden whom the angels name Lenore—\n",
      "\n",
      "Clasp a rare and radiant maiden whom the angels name Lenore.”\n",
      "\n",
      "            Quoth the Raven “Nevermore.”\n",
      "\n",
      "\n",
      "\n",
      "    “Be that word our sign of parting, bird or fiend!” I shrieked, upstarting—\n",
      "\n",
      "“Get thee back into the tempest and the Night’s Plutonian shore!\n",
      "\n",
      "    Leave no black plume as a token of that lie thy soul hath spoken!\n",
      "\n",
      "    Leave my loneliness unbroken!—quit the bust above my door!\n",
      "\n",
      "Take thy beak from out my heart, and take thy form from off my door!”\n",
      "\n",
      "            Quoth the Raven “Nevermore.”\n",
      "\n",
      "\n",
      "\n",
      "    And the Raven, never flitting, still is sitting, still is sitting\n",
      "\n",
      "On the pallid bust of Pallas just above my chamber door;\n",
      "\n",
      "    And his eyes have all the seeming of a demon’s that is dreaming,\n",
      "\n",
      "    And the lamp-light o’er him streaming throws his shadow on the floor;\n",
      "\n",
      "And my soul from out that shadow that lies floating on the floor\n",
      "\n",
      "            Shall be lifted—nevermore!\n"
     ]
    }
   ],
   "source": [
    "infile = open('raven.txt', 'r') # makes our fileio object\n",
    "\n",
    "for line in infile:\n",
    "    print(line)\n",
    "\n",
    "infile.close()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Here we're only printing out the line of text.  Which is a great place to start, because we've isolated them and can manipulate each."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Step 2: do the thing you want to the line\n",
    "\n",
    "Now that we have each line of text isolated, we can act on them.  Take a moment to identify where each line of text is being held in the code.  Thus answering the question, which variable should be passed into `len()`?\n",
    "\n",
    "The answer is `line`.  This is our iterable variable, and holds each line of text, one at a time.  This is the one that we should manipulate for the content.\n",
    "\n",
    "So let's add that into the above code."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "67\n",
      "57\n",
      "67\n",
      "59\n",
      "62\n",
      "41\n",
      "1\n",
      "60\n",
      "64\n",
      "63\n",
      "65\n",
      "61\n",
      "40\n",
      "1\n",
      "67\n",
      "64\n",
      "69\n",
      "63\n",
      "59\n",
      "42\n",
      "1\n",
      "64\n",
      "60\n",
      "67\n",
      "65\n",
      "66\n",
      "45\n",
      "1\n",
      "76\n",
      "64\n",
      "67\n",
      "69\n",
      "65\n",
      "42\n",
      "1\n",
      "66\n",
      "58\n",
      "70\n",
      "67\n",
      "58\n",
      "45\n",
      "1\n",
      "72\n",
      "62\n",
      "72\n",
      "67\n",
      "58\n",
      "48\n",
      "1\n",
      "62\n",
      "59\n",
      "75\n",
      "65\n",
      "65\n",
      "41\n",
      "1\n",
      "70\n",
      "56\n",
      "59\n",
      "66\n",
      "63\n",
      "43\n",
      "1\n",
      "65\n",
      "63\n",
      "69\n",
      "73\n",
      "64\n",
      "44\n",
      "1\n",
      "63\n",
      "65\n",
      "61\n",
      "70\n",
      "56\n",
      "35\n",
      "1\n",
      "61\n",
      "73\n",
      "62\n",
      "63\n",
      "67\n",
      "43\n",
      "1\n",
      "63\n",
      "62\n",
      "65\n",
      "69\n",
      "66\n",
      "44\n",
      "1\n",
      "73\n",
      "64\n",
      "81\n",
      "62\n",
      "65\n",
      "41\n",
      "1\n",
      "73\n",
      "66\n",
      "63\n",
      "61\n",
      "63\n",
      "41\n",
      "1\n",
      "72\n",
      "62\n",
      "68\n",
      "65\n",
      "62\n",
      "41\n",
      "1\n",
      "79\n",
      "65\n",
      "70\n",
      "63\n",
      "70\n",
      "41\n",
      "1\n",
      "70\n",
      "57\n",
      "69\n",
      "74\n",
      "65\n",
      "38\n"
     ]
    }
   ],
   "source": [
    "infile = open('raven.txt', 'r') # makes our fileio object\n",
    "\n",
    "for line in infile:\n",
    "    line_length = len(line)\n",
    "    print(line_length)\n",
    "\n",
    "infile.close()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "So here we are running `line` through `len()` and saving that returned value as `line_length`.  Then we are printing that out.\n",
    "\n",
    "Printing the value is a great place to start because it allows you to visualize what's going on with the results.  We could also print both the line and the length, so we can see the original and the results.  This helps us further check out work and confirm that we are accomplishing what we wanted.  "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "67     Once upon a midnight dreary, while I pondered, weak and weary,\n",
      "\n",
      "57 Over many a quaint and curious volume of forgotten lore—\n",
      "\n",
      "67     While I nodded, nearly napping, suddenly there came a tapping,\n",
      "\n",
      "59 As of some one gently rapping, rapping at my chamber door.\n",
      "\n",
      "62 “’Tis some visitor,” I muttered, “tapping at my chamber door—\n",
      "\n",
      "41             Only this and nothing more.”\n",
      "\n",
      "1 \n",
      "\n",
      "60     Ah, distinctly I remember it was in the bleak December;\n",
      "\n",
      "64 And each separate dying ember wrought its ghost upon the floor.\n",
      "\n",
      "63     Eagerly I wished the morrow;—vainly I had sought to borrow\n",
      "\n",
      "65     From my books surcease of sorrow—sorrow for the lost Lenore—\n",
      "\n",
      "61 For the rare and radiant maiden whom the angels name Lenore—\n",
      "\n",
      "40             Nameless here for evermore.\n",
      "\n",
      "1 \n",
      "\n",
      "67     And the silken, sad, uncertain rustling of each purple curtain\n",
      "\n",
      "64 Thrilled me—filled me with fantastic terrors never felt before;\n",
      "\n",
      "69     So that now, to still the beating of my heart, I stood repeating\n",
      "\n",
      "63     “’Tis some visitor entreating entrance at my chamber door—\n",
      "\n",
      "59 Some late visitor entreating entrance at my chamber door;—\n",
      "\n",
      "42             This it is and nothing more.”\n",
      "\n",
      "1 \n",
      "\n",
      "64     Presently my soul grew stronger; hesitating then no longer,\n",
      "\n",
      "60 “Sir,” said I, “or Madam, truly your forgiveness I implore;\n",
      "\n",
      "67     But the fact is I was napping, and so gently you came rapping,\n",
      "\n",
      "65     And so faintly you came tapping, tapping at my chamber door,\n",
      "\n",
      "66 That I scarce was sure I heard you”—here I opened wide the door;—\n",
      "\n",
      "45             Darkness there and nothing more.\n",
      "\n",
      "1 \n",
      "\n",
      "76     Deep into that darkness peering, long I stood there wondering, fearing,\n",
      "\n",
      "64 Doubting, dreaming dreams no mortal ever dared to dream before;\n",
      "\n",
      "67     But the silence was unbroken, and the stillness gave no token,\n",
      "\n",
      "69     And the only word there spoken was the whispered word, “Lenore?”\n",
      "\n",
      "65 This I whispered, and an echo murmured back the word, “Lenore!”—\n",
      "\n",
      "42             Merely this and nothing more.\n",
      "\n",
      "1 \n",
      "\n",
      "66     Back into the chamber turning, all my soul within me burning,\n",
      "\n",
      "58 Soon again I heard a tapping somewhat louder than before.\n",
      "\n",
      "70     “Surely,” said I, “surely that is something at my window lattice;\n",
      "\n",
      "67       Let me see, then, what thereat is, and this mystery explore—\n",
      "\n",
      "58 Let my heart be still a moment and this mystery explore;—\n",
      "\n",
      "45             ’Tis the wind and nothing more!”\n",
      "\n",
      "1 \n",
      "\n",
      "72     Open here I flung the shutter, when, with many a flirt and flutter,\n",
      "\n",
      "62 In there stepped a stately Raven of the saintly days of yore;\n",
      "\n",
      "72     Not the least obeisance made he; not a minute stopped or stayed he;\n",
      "\n",
      "67     But, with mien of lord or lady, perched above my chamber door—\n",
      "\n",
      "58 Perched upon a bust of Pallas just above my chamber door—\n",
      "\n",
      "48             Perched, and sat, and nothing more.\n",
      "\n",
      "1 \n",
      "\n",
      "62     Then this ebony bird beguiling my sad fancy into smiling,\n",
      "\n",
      "59 By the grave and stern decorum of the countenance it wore,\n",
      "\n",
      "75 “Though thy crest be shorn and shaven, thou,” I said, “art sure no craven,\n",
      "\n",
      "65 Ghastly grim and ancient Raven wandering from the Nightly shore—\n",
      "\n",
      "65 Tell me what thy lordly name is on the Night’s Plutonian shore!”\n",
      "\n",
      "41             Quoth the Raven “Nevermore.”\n",
      "\n",
      "1 \n",
      "\n",
      "70     Much I marvelled this ungainly fowl to hear discourse so plainly,\n",
      "\n",
      "56 Though its answer little meaning—little relevancy bore;\n",
      "\n",
      "59     For we cannot help agreeing that no living human being\n",
      "\n",
      "66     Ever yet was blessed with seeing bird above his chamber door—\n",
      "\n",
      "63 Bird or beast upon the sculptured bust above his chamber door,\n",
      "\n",
      "43             With such name as “Nevermore.”\n",
      "\n",
      "1 \n",
      "\n",
      "65     But the Raven, sitting lonely on the placid bust, spoke only\n",
      "\n",
      "63 That one word, as if his soul in that one word he did outpour.\n",
      "\n",
      "69     Nothing farther then he uttered—not a feather then he fluttered—\n",
      "\n",
      "73     Till I scarcely more than muttered “Other friends have flown before—\n",
      "\n",
      "64 On the morrow he will leave me, as my Hopes have flown before.”\n",
      "\n",
      "44             Then the bird said “Nevermore.”\n",
      "\n",
      "1 \n",
      "\n",
      "63     Startled at the stillness broken by reply so aptly spoken,\n",
      "\n",
      "65 “Doubtless,” said I, “what it utters is its only stock and store\n",
      "\n",
      "61     Caught from some unhappy master whom unmerciful Disaster\n",
      "\n",
      "70     Followed fast and followed faster till his songs one burden bore—\n",
      "\n",
      "56 Till the dirges of his Hope that melancholy burden bore\n",
      "\n",
      "35             Of ‘Never—nevermore’.”\n",
      "\n",
      "1 \n",
      "\n",
      "61     But the Raven still beguiling all my fancy into smiling,\n",
      "\n",
      "73 Straight I wheeled a cushioned seat in front of bird, and bust and door;\n",
      "\n",
      "62     Then, upon the velvet sinking, I betook myself to linking\n",
      "\n",
      "63     Fancy unto fancy, thinking what this ominous bird of yore—\n",
      "\n",
      "67 What this grim, ungainly, ghastly, gaunt, and ominous bird of yore\n",
      "\n",
      "43             Meant in croaking “Nevermore.”\n",
      "\n",
      "1 \n",
      "\n",
      "63     This I sat engaged in guessing, but no syllable expressing\n",
      "\n",
      "62 To the fowl whose fiery eyes now burned into my bosom’s core;\n",
      "\n",
      "65     This and more I sat divining, with my head at ease reclining\n",
      "\n",
      "69     On the cushion’s velvet lining that the lamp-light gloated o’er,\n",
      "\n",
      "66 But whose velvet-violet lining with the lamp-light gloating o’er,\n",
      "\n",
      "44             She shall press, ah, nevermore!\n",
      "\n",
      "1 \n",
      "\n",
      "73     Then, methought, the air grew denser, perfumed from an unseen censer\n",
      "\n",
      "64 Swung by Seraphim whose foot-falls tinkled on the tufted floor.\n",
      "\n",
      "81     “Wretch,” I cried, “thy God hath lent thee—by these angels he hath sent thee\n",
      "\n",
      "62     Respite—respite and nepenthe from thy memories of Lenore;\n",
      "\n",
      "65 Quaff, oh quaff this kind nepenthe and forget this lost Lenore!”\n",
      "\n",
      "41             Quoth the Raven “Nevermore.”\n",
      "\n",
      "1 \n",
      "\n",
      "73     “Prophet!” said I, “thing of evil!—prophet still, if bird or devil!—\n",
      "\n",
      "66 Whether Tempter sent, or whether tempest tossed thee here ashore,\n",
      "\n",
      "63     Desolate yet all undaunted, on this desert land enchanted—\n",
      "\n",
      "61     On this home by Horror haunted—tell me truly, I implore—\n",
      "\n",
      "63 Is there—is there balm in Gilead?—tell me—tell me, I implore!”\n",
      "\n",
      "41             Quoth the Raven “Nevermore.”\n",
      "\n",
      "1 \n",
      "\n",
      "72     “Prophet!” said I, “thing of evil!—prophet still, if bird or devil!\n",
      "\n",
      "62 By that Heaven that bends above us—by that God we both adore—\n",
      "\n",
      "68     Tell this soul with sorrow laden if, within the distant Aidenn,\n",
      "\n",
      "65     It shall clasp a sainted maiden whom the angels name Lenore—\n",
      "\n",
      "62 Clasp a rare and radiant maiden whom the angels name Lenore.”\n",
      "\n",
      "41             Quoth the Raven “Nevermore.”\n",
      "\n",
      "1 \n",
      "\n",
      "79     “Be that word our sign of parting, bird or fiend!” I shrieked, upstarting—\n",
      "\n",
      "65 “Get thee back into the tempest and the Night’s Plutonian shore!\n",
      "\n",
      "70     Leave no black plume as a token of that lie thy soul hath spoken!\n",
      "\n",
      "63     Leave my loneliness unbroken!—quit the bust above my door!\n",
      "\n",
      "70 Take thy beak from out my heart, and take thy form from off my door!”\n",
      "\n",
      "41             Quoth the Raven “Nevermore.”\n",
      "\n",
      "1 \n",
      "\n",
      "70     And the Raven, never flitting, still is sitting, still is sitting\n",
      "\n",
      "57 On the pallid bust of Pallas just above my chamber door;\n",
      "\n",
      "69     And his eyes have all the seeming of a demon’s that is dreaming,\n",
      "\n",
      "74     And the lamp-light o’er him streaming throws his shadow on the floor;\n",
      "\n",
      "65 And my soul from out that shadow that lies floating on the floor\n",
      "\n",
      "38             Shall be lifted—nevermore!\n"
     ]
    }
   ],
   "source": [
    "infile = open('raven.txt', 'r') # makes our fileio object\n",
    "\n",
    "for line in infile:\n",
    "    line_length = len(line)\n",
    "    print(line_length, line)\n",
    "\n",
    "infile.close()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "I'm printing the length first before the line content for readability.  The lines vary in length so much that the length numbers wouldn't line up in a nice way. Having them come first makes it easier for us to glance at things.\n",
    "\n",
    "## Step 3:\n",
    "\n",
    "Write out the results.\n",
    "\n",
    "Now that we have the output that we want, we can add our outfile into the mix."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "infile = open('raven.txt', 'r') # makes our fileio object\n",
    "outfile = open('raven_line_length.txt', 'w')\n",
    "\n",
    "for line in infile:\n",
    "    line_length = len(line)\n",
    "    print(line_length, file = outfile) # changing this back to just the line length as well\n",
    "\n",
    "infile.close()\n",
    "outfile.close()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "And we're done!  This illustrates the three stages of adapting a piece of code to batch processing.  First, read in the file.  Second, act on each line of text, getting the result for that line that you need.  Print them out to check you work.  Then third, redirect that print to an outfile.\n",
    "\n",
    "## another perspective\n",
    "\n",
    "This code looks quite clean because we have a function that is doing the work for us.  So we don't have a lot of nesting or extra stuff inside our for loop reading the file.\n",
    "\n",
    "However, if we don't have a function, but we do have a chunk of code that operates on a single line of text, we can still use that.  It'll get moved in under the for loop that's reading in the file.\n",
    "\n",
    "So if we have something that's stripping out while space and reversing the word order of the text, we can do something like this:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "dreaming, is that demon’s a of seeming the all have eyes his And\n"
     ]
    }
   ],
   "source": [
    "# start with the code that works\n",
    "\n",
    "line = \"     And his eyes have all the seeming of a demon’s that is dreaming,\"\n",
    "\n",
    "clean = line.strip()\n",
    "\n",
    "reverse = []\n",
    "\n",
    "splitwords = clean.split()\n",
    "\n",
    "for pos in range(len(splitwords) -1, -1, -1): # run backwards from last to first\n",
    "    word = splitwords[pos]\n",
    "    reverse.append(word)\n",
    "    \n",
    "reversed_line = \" \".join(reverse)\n",
    "print(reversed_line)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "So here we have the reversed sentence, and this code required our use of a for loop.  We can apply this to all of the raven via scooting this under the for loop going over each line.\n",
    "\n",
    "This means we'll have something like:\n",
    "\n",
    "``` python\n",
    "for line in file:\n",
    "    do stuff\n",
    "    for word in line:\n",
    "        do more stuff\n",
    "    print(go to the outfile)\n",
    "```\n",
    "\n",
    "Let's follow our previous three steps that we used for doing this with a function.  Instead of building this up and adding the function, we're going to build it up and add this code in."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 32,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "    Once upon a midnight dreary, while I pondered, weak and weary,\n",
      "\n",
      "Over many a quaint and curious volume of forgotten lore—\n",
      "\n",
      "    While I nodded, nearly napping, suddenly there came a tapping,\n",
      "\n",
      "As of some one gently rapping, rapping at my chamber door.\n",
      "\n",
      "“’Tis some visitor,” I muttered, “tapping at my chamber door—\n",
      "\n",
      "            Only this and nothing more.”\n",
      "\n",
      "\n",
      "\n",
      "    Ah, distinctly I remember it was in the bleak December;\n",
      "\n",
      "And each separate dying ember wrought its ghost upon the floor.\n",
      "\n",
      "    Eagerly I wished the morrow;—vainly I had sought to borrow\n",
      "\n",
      "    From my books surcease of sorrow—sorrow for the lost Lenore—\n",
      "\n",
      "For the rare and radiant maiden whom the angels name Lenore—\n",
      "\n",
      "            Nameless here for evermore.\n",
      "\n",
      "\n",
      "\n",
      "    And the silken, sad, uncertain rustling of each purple curtain\n",
      "\n",
      "Thrilled me—filled me with fantastic terrors never felt before;\n",
      "\n",
      "    So that now, to still the beating of my heart, I stood repeating\n",
      "\n",
      "    “’Tis some visitor entreating entrance at my chamber door—\n",
      "\n",
      "Some late visitor entreating entrance at my chamber door;—\n",
      "\n",
      "            This it is and nothing more.”\n",
      "\n",
      "\n",
      "\n",
      "    Presently my soul grew stronger; hesitating then no longer,\n",
      "\n",
      "“Sir,” said I, “or Madam, truly your forgiveness I implore;\n",
      "\n",
      "    But the fact is I was napping, and so gently you came rapping,\n",
      "\n",
      "    And so faintly you came tapping, tapping at my chamber door,\n",
      "\n",
      "That I scarce was sure I heard you”—here I opened wide the door;—\n",
      "\n",
      "            Darkness there and nothing more.\n",
      "\n",
      "\n",
      "\n",
      "    Deep into that darkness peering, long I stood there wondering, fearing,\n",
      "\n",
      "Doubting, dreaming dreams no mortal ever dared to dream before;\n",
      "\n",
      "    But the silence was unbroken, and the stillness gave no token,\n",
      "\n",
      "    And the only word there spoken was the whispered word, “Lenore?”\n",
      "\n",
      "This I whispered, and an echo murmured back the word, “Lenore!”—\n",
      "\n",
      "            Merely this and nothing more.\n",
      "\n",
      "\n",
      "\n",
      "    Back into the chamber turning, all my soul within me burning,\n",
      "\n",
      "Soon again I heard a tapping somewhat louder than before.\n",
      "\n",
      "    “Surely,” said I, “surely that is something at my window lattice;\n",
      "\n",
      "      Let me see, then, what thereat is, and this mystery explore—\n",
      "\n",
      "Let my heart be still a moment and this mystery explore;—\n",
      "\n",
      "            ’Tis the wind and nothing more!”\n",
      "\n",
      "\n",
      "\n",
      "    Open here I flung the shutter, when, with many a flirt and flutter,\n",
      "\n",
      "In there stepped a stately Raven of the saintly days of yore;\n",
      "\n",
      "    Not the least obeisance made he; not a minute stopped or stayed he;\n",
      "\n",
      "    But, with mien of lord or lady, perched above my chamber door—\n",
      "\n",
      "Perched upon a bust of Pallas just above my chamber door—\n",
      "\n",
      "            Perched, and sat, and nothing more.\n",
      "\n",
      "\n",
      "\n",
      "    Then this ebony bird beguiling my sad fancy into smiling,\n",
      "\n",
      "By the grave and stern decorum of the countenance it wore,\n",
      "\n",
      "“Though thy crest be shorn and shaven, thou,” I said, “art sure no craven,\n",
      "\n",
      "Ghastly grim and ancient Raven wandering from the Nightly shore—\n",
      "\n",
      "Tell me what thy lordly name is on the Night’s Plutonian shore!”\n",
      "\n",
      "            Quoth the Raven “Nevermore.”\n",
      "\n",
      "\n",
      "\n",
      "    Much I marvelled this ungainly fowl to hear discourse so plainly,\n",
      "\n",
      "Though its answer little meaning—little relevancy bore;\n",
      "\n",
      "    For we cannot help agreeing that no living human being\n",
      "\n",
      "    Ever yet was blessed with seeing bird above his chamber door—\n",
      "\n",
      "Bird or beast upon the sculptured bust above his chamber door,\n",
      "\n",
      "            With such name as “Nevermore.”\n",
      "\n",
      "\n",
      "\n",
      "    But the Raven, sitting lonely on the placid bust, spoke only\n",
      "\n",
      "That one word, as if his soul in that one word he did outpour.\n",
      "\n",
      "    Nothing farther then he uttered—not a feather then he fluttered—\n",
      "\n",
      "    Till I scarcely more than muttered “Other friends have flown before—\n",
      "\n",
      "On the morrow he will leave me, as my Hopes have flown before.”\n",
      "\n",
      "            Then the bird said “Nevermore.”\n",
      "\n",
      "\n",
      "\n",
      "    Startled at the stillness broken by reply so aptly spoken,\n",
      "\n",
      "“Doubtless,” said I, “what it utters is its only stock and store\n",
      "\n",
      "    Caught from some unhappy master whom unmerciful Disaster\n",
      "\n",
      "    Followed fast and followed faster till his songs one burden bore—\n",
      "\n",
      "Till the dirges of his Hope that melancholy burden bore\n",
      "\n",
      "            Of ‘Never—nevermore’.”\n",
      "\n",
      "\n",
      "\n",
      "    But the Raven still beguiling all my fancy into smiling,\n",
      "\n",
      "Straight I wheeled a cushioned seat in front of bird, and bust and door;\n",
      "\n",
      "    Then, upon the velvet sinking, I betook myself to linking\n",
      "\n",
      "    Fancy unto fancy, thinking what this ominous bird of yore—\n",
      "\n",
      "What this grim, ungainly, ghastly, gaunt, and ominous bird of yore\n",
      "\n",
      "            Meant in croaking “Nevermore.”\n",
      "\n",
      "\n",
      "\n",
      "    This I sat engaged in guessing, but no syllable expressing\n",
      "\n",
      "To the fowl whose fiery eyes now burned into my bosom’s core;\n",
      "\n",
      "    This and more I sat divining, with my head at ease reclining\n",
      "\n",
      "    On the cushion’s velvet lining that the lamp-light gloated o’er,\n",
      "\n",
      "But whose velvet-violet lining with the lamp-light gloating o’er,\n",
      "\n",
      "            She shall press, ah, nevermore!\n",
      "\n",
      "\n",
      "\n",
      "    Then, methought, the air grew denser, perfumed from an unseen censer\n",
      "\n",
      "Swung by Seraphim whose foot-falls tinkled on the tufted floor.\n",
      "\n",
      "    “Wretch,” I cried, “thy God hath lent thee—by these angels he hath sent thee\n",
      "\n",
      "    Respite—respite and nepenthe from thy memories of Lenore;\n",
      "\n",
      "Quaff, oh quaff this kind nepenthe and forget this lost Lenore!”\n",
      "\n",
      "            Quoth the Raven “Nevermore.”\n",
      "\n",
      "\n",
      "\n",
      "    “Prophet!” said I, “thing of evil!—prophet still, if bird or devil!—\n",
      "\n",
      "Whether Tempter sent, or whether tempest tossed thee here ashore,\n",
      "\n",
      "    Desolate yet all undaunted, on this desert land enchanted—\n",
      "\n",
      "    On this home by Horror haunted—tell me truly, I implore—\n",
      "\n",
      "Is there—is there balm in Gilead?—tell me—tell me, I implore!”\n",
      "\n",
      "            Quoth the Raven “Nevermore.”\n",
      "\n",
      "\n",
      "\n",
      "    “Prophet!” said I, “thing of evil!—prophet still, if bird or devil!\n",
      "\n",
      "By that Heaven that bends above us—by that God we both adore—\n",
      "\n",
      "    Tell this soul with sorrow laden if, within the distant Aidenn,\n",
      "\n",
      "    It shall clasp a sainted maiden whom the angels name Lenore—\n",
      "\n",
      "Clasp a rare and radiant maiden whom the angels name Lenore.”\n",
      "\n",
      "            Quoth the Raven “Nevermore.”\n",
      "\n",
      "\n",
      "\n",
      "    “Be that word our sign of parting, bird or fiend!” I shrieked, upstarting—\n",
      "\n",
      "“Get thee back into the tempest and the Night’s Plutonian shore!\n",
      "\n",
      "    Leave no black plume as a token of that lie thy soul hath spoken!\n",
      "\n",
      "    Leave my loneliness unbroken!—quit the bust above my door!\n",
      "\n",
      "Take thy beak from out my heart, and take thy form from off my door!”\n",
      "\n",
      "            Quoth the Raven “Nevermore.”\n",
      "\n",
      "\n",
      "\n",
      "    And the Raven, never flitting, still is sitting, still is sitting\n",
      "\n",
      "On the pallid bust of Pallas just above my chamber door;\n",
      "\n",
      "    And his eyes have all the seeming of a demon’s that is dreaming,\n",
      "\n",
      "    And the lamp-light o’er him streaming throws his shadow on the floor;\n",
      "\n",
      "And my soul from out that shadow that lies floating on the floor\n",
      "\n",
      "            Shall be lifted—nevermore!\n"
     ]
    }
   ],
   "source": [
    "# exactly the same code that we went with before.\n",
    "infile = open('raven.txt', 'r')\n",
    "\n",
    "for line in infile:\n",
    "    print(line)\n",
    "    \n",
    "infile.close()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Now we're going to move the bulk of our code over, and print out the results for each line.  I'm going to copy over all of the code, except for 'line = ...' because the line is being defined within the for loop.\n",
    "\n",
    "The code below has exactly our previous code, just without that first line. Because my variable name matched what I used as my iterable, I don't need to fuss over where the split is happening."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 33,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "weary, and weak pondered, I while dreary, midnight a upon Once\n",
      "lore— forgotten of volume curious and quaint a many Over\n",
      "tapping, a came there suddenly napping, nearly nodded, I While\n",
      "door. chamber my at rapping rapping, gently one some of As\n",
      "door— chamber my at “tapping muttered, I visitor,” some “’Tis\n",
      "more.” nothing and this Only\n",
      "\n",
      "December; bleak the in was it remember I distinctly Ah,\n",
      "floor. the upon ghost its wrought ember dying separate each And\n",
      "borrow to sought had I morrow;—vainly the wished I Eagerly\n",
      "Lenore— lost the for sorrow—sorrow of surcease books my From\n",
      "Lenore— name angels the whom maiden radiant and rare the For\n",
      "evermore. for here Nameless\n",
      "\n",
      "curtain purple each of rustling uncertain sad, silken, the And\n",
      "before; felt never terrors fantastic with me me—filled Thrilled\n",
      "repeating stood I heart, my of beating the still to now, that So\n",
      "door— chamber my at entrance entreating visitor some “’Tis\n",
      "door;— chamber my at entrance entreating visitor late Some\n",
      "more.” nothing and is it This\n",
      "\n",
      "longer, no then hesitating stronger; grew soul my Presently\n",
      "implore; I forgiveness your truly Madam, “or I, said “Sir,”\n",
      "rapping, came you gently so and napping, was I is fact the But\n",
      "door, chamber my at tapping tapping, came you faintly so And\n",
      "door;— the wide opened I you”—here heard I sure was scarce I That\n",
      "more. nothing and there Darkness\n",
      "\n",
      "fearing, wondering, there stood I long peering, darkness that into Deep\n",
      "before; dream to dared ever mortal no dreams dreaming Doubting,\n",
      "token, no gave stillness the and unbroken, was silence the But\n",
      "“Lenore?” word, whispered the was spoken there word only the And\n",
      "“Lenore!”— word, the back murmured echo an and whispered, I This\n",
      "more. nothing and this Merely\n",
      "\n",
      "burning, me within soul my all turning, chamber the into Back\n",
      "before. than louder somewhat tapping a heard I again Soon\n",
      "lattice; window my at something is that “surely I, said “Surely,”\n",
      "explore— mystery this and is, thereat what then, see, me Let\n",
      "explore;— mystery this and moment a still be heart my Let\n",
      "more!” nothing and wind the ’Tis\n",
      "\n",
      "flutter, and flirt a many with when, shutter, the flung I here Open\n",
      "yore; of days saintly the of Raven stately a stepped there In\n",
      "he; stayed or stopped minute a not he; made obeisance least the Not\n",
      "door— chamber my above perched lady, or lord of mien with But,\n",
      "door— chamber my above just Pallas of bust a upon Perched\n",
      "more. nothing and sat, and Perched,\n",
      "\n",
      "smiling, into fancy sad my beguiling bird ebony this Then\n",
      "wore, it countenance the of decorum stern and grave the By\n",
      "craven, no sure “art said, I thou,” shaven, and shorn be crest thy “Though\n",
      "shore— Nightly the from wandering Raven ancient and grim Ghastly\n",
      "shore!” Plutonian Night’s the on is name lordly thy what me Tell\n",
      "“Nevermore.” Raven the Quoth\n",
      "\n",
      "plainly, so discourse hear to fowl ungainly this marvelled I Much\n",
      "bore; relevancy meaning—little little answer its Though\n",
      "being human living no that agreeing help cannot we For\n",
      "door— chamber his above bird seeing with blessed was yet Ever\n",
      "door, chamber his above bust sculptured the upon beast or Bird\n",
      "“Nevermore.” as name such With\n",
      "\n",
      "only spoke bust, placid the on lonely sitting Raven, the But\n",
      "outpour. did he word one that in soul his if as word, one That\n",
      "fluttered— he then feather a uttered—not he then farther Nothing\n",
      "before— flown have friends “Other muttered than more scarcely I Till\n",
      "before.” flown have Hopes my as me, leave will he morrow the On\n",
      "“Nevermore.” said bird the Then\n",
      "\n",
      "spoken, aptly so reply by broken stillness the at Startled\n",
      "store and stock only its is utters it “what I, said “Doubtless,”\n",
      "Disaster unmerciful whom master unhappy some from Caught\n",
      "bore— burden one songs his till faster followed and fast Followed\n",
      "bore burden melancholy that Hope his of dirges the Till\n",
      "‘Never—nevermore’.” Of\n",
      "\n",
      "smiling, into fancy my all beguiling still Raven the But\n",
      "door; and bust and bird, of front in seat cushioned a wheeled I Straight\n",
      "linking to myself betook I sinking, velvet the upon Then,\n",
      "yore— of bird ominous this what thinking fancy, unto Fancy\n",
      "yore of bird ominous and gaunt, ghastly, ungainly, grim, this What\n",
      "“Nevermore.” croaking in Meant\n",
      "\n",
      "expressing syllable no but guessing, in engaged sat I This\n",
      "core; bosom’s my into burned now eyes fiery whose fowl the To\n",
      "reclining ease at head my with divining, sat I more and This\n",
      "o’er, gloated lamp-light the that lining velvet cushion’s the On\n",
      "o’er, gloating lamp-light the with lining velvet-violet whose But\n",
      "nevermore! ah, press, shall She\n",
      "\n",
      "censer unseen an from perfumed denser, grew air the methought, Then,\n",
      "floor. tufted the on tinkled foot-falls whose Seraphim by Swung\n",
      "thee sent hath he angels these thee—by lent hath God “thy cried, I “Wretch,”\n",
      "Lenore; of memories thy from nepenthe and Respite—respite\n",
      "Lenore!” lost this forget and nepenthe kind this quaff oh Quaff,\n",
      "“Nevermore.” Raven the Quoth\n",
      "\n",
      "devil!— or bird if still, evil!—prophet of “thing I, said “Prophet!”\n",
      "ashore, here thee tossed tempest whether or sent, Tempter Whether\n",
      "enchanted— land desert this on undaunted, all yet Desolate\n",
      "implore— I truly, me haunted—tell Horror by home this On\n",
      "implore!” I me, me—tell Gilead?—tell in balm there there—is Is\n",
      "“Nevermore.” Raven the Quoth\n",
      "\n",
      "devil! or bird if still, evil!—prophet of “thing I, said “Prophet!”\n",
      "adore— both we God that us—by above bends that Heaven that By\n",
      "Aidenn, distant the within if, laden sorrow with soul this Tell\n",
      "Lenore— name angels the whom maiden sainted a clasp shall It\n",
      "Lenore.” name angels the whom maiden radiant and rare a Clasp\n",
      "“Nevermore.” Raven the Quoth\n",
      "\n",
      "upstarting— shrieked, I fiend!” or bird parting, of sign our word that “Be\n",
      "shore! Plutonian Night’s the and tempest the into back thee “Get\n",
      "spoken! hath soul thy lie that of token a as plume black no Leave\n",
      "door! my above bust the unbroken!—quit loneliness my Leave\n",
      "door!” my off from form thy take and heart, my out from beak thy Take\n",
      "“Nevermore.” Raven the Quoth\n",
      "\n",
      "sitting is still sitting, is still flitting, never Raven, the And\n",
      "door; chamber my above just Pallas of bust pallid the On\n",
      "dreaming, is that demon’s a of seeming the all have eyes his And\n",
      "floor; the on shadow his throws streaming him o’er lamp-light the And\n",
      "floor the on floating lies that shadow that out from soul my And\n",
      "lifted—nevermore! be Shall\n"
     ]
    }
   ],
   "source": [
    "infile = open('raven.txt', 'r')\n",
    "\n",
    "for line in infile:\n",
    "    clean = line.strip()\n",
    "\n",
    "    reverse = []\n",
    "\n",
    "    splitwords = clean.split()\n",
    "\n",
    "    for pos in range(len(splitwords) -1, -1, -1): # run backwards from last to first\n",
    "        word = splitwords[pos]\n",
    "        reverse.append(word)\n",
    "\n",
    "    reversed_line = \" \".join(reverse)\n",
    "    print(reversed_line)\n",
    "    \n",
    "infile.close()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Great, so that's working over each line and printing it out.  Now, we can add the outfile."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 34,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "infile = open('raven.txt', 'r')\n",
    "outfile = open('raven_reversed.txt', 'w')\n",
    "\n",
    "for line in infile:\n",
    "    clean = line.strip()\n",
    "\n",
    "    reverse = []\n",
    "\n",
    "    splitwords = clean.split()\n",
    "\n",
    "    for pos in range(len(splitwords) -1, -1, -1): # run backwards from last to first\n",
    "        word = splitwords[pos]\n",
    "        reverse.append(word)\n",
    "\n",
    "    reversed_line = \" \".join(reverse)\n",
    "    print(reversed_line, file = outfile)\n",
    "    \n",
    "infile.close()\n",
    "outfile.close()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": []
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": []
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": []
  }
 ],
 "metadata": {
  "anaconda-cloud": {},
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.6.1"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 1
}