How to code? — 2

Don’t code until you know the problem it will solve.

Read post-1 to get the background.

Coding is talking to computers in the language they understand. Phew, that’s so cool, isn’t it? We are going to use python as it is relatively easy on syntax and has a lot of ready made libraries for a lot of things.

But before we tell the computer to do things for us, we have to spend time to understand what we need in as much elaborate terms as possible. In software engineering we call this as ‘Requirement analysis’.

Most of the time, software is developed for people who may not know how to code. So there is a gap between what they state as a problem to be solved and what needs to be realized by code.

We began by stating the need to backup files on a PC as follows

I have a pain of backing up my folders on my PC onto a external drive. Every time I have to copy these folders, wait for the whole thing to complete and then remove the old copy from the external disk to free up its space.

Now, we need to expand on this to understand what is conveyed and what is not conveyed as well. This is done by asking questions about each aspect of the problem statement to expand the requirements. This may show us the scope of our needs. Sometimes, we may find that the problem is much bigger than we thought. And we may decide to keep it simple by drawing boundaries around the scope. Sometimes this analysis may bring in more clarity on the type of solution or the approach we would take. Effectively you go on a no holds barred approach in thinking everything. Then you can identify which ones you want to go forward and which ones you want to drop off. Essentially this will draw the scope or boundary of what we want to solve.

For example, ‘I have a pain of backing up my folders on my PC onto a external drive’ conveys the one line need. If you translate it, it is, Write a program to copy folders on a PC to a external drive.

Now this can bring questions like,

• Which folders do we want to copy ?
• Are they always the same folder every time bakitup runs? or the user can select the folders to be backed up? or is it a one time selection and bakitup copies the same set of folders every time it runs?
• Should the program ask the user every time it runs? Probably not. Because, backup should be an automatic process that should happen without much user intervention. So once the user identifies the source folders to backup, the first time, then on wards it should backup the same set of folders until the user changes it next time.
• Now since we brought in the idea of ‘automatically running without user intervention’, it brings the question of how often it should run. This again can be whenever the user powers on his PC. Or the user can set a frequency for periodic backups.
• When user runs bakitup for the first time, the external drive should be connected and identified as the destination where the files have to be copied by the user.
• When bakitup runs periodically, if it finds files to be backed up in the configured folders, it must check if the external drive is connected. If not, it should notify the user to connect the same.
• This means when bakitup runs automatically, it must find files in the source folder that have changed since the last backup, to compile a list of files that need to be backed up. This should trigger the check to see if the external drive is connected or not and then result in notifying the user if it is not connected.
• It is possible that while the backup is running, the user shuts down the PC. The back up may be half way through a file. This can result in inconsistent data between the source and the target. To avoid this, before copying a file, the destination drive should be copied with a meta file that has the name of the file that is about to be copied and probably the size of the file to be copied and after copying, remove that meta file, once the size is validated to be the same. In case the user shuts down before removing the meta file, next time when bakitup runs, it will copy the file again.
• This also brings the question as to how the bakitup will record the last backed up timestamp, if the user shuts down the PC while backup is going on. To properly address this, suppose when bakitup runs and finds X number of files to be backed up. It should sort this list from earliest to latest timestamp and record the timestamp of the file in the meta file above. After a file is copied to the destination and before the meta file is removed, it must record the file’s timestamp in another file on the destination which should serve as the last backed up timestamp. It is possible that the user may shutdown the PC exactly before recording the timestamp of the file copied. But then, we have the meta file which is not yet removed, which means the next backup will restart from that file and all others equal or of later timestamp.

Now you see how we brainstorm that one line into a number of requirements, sometimes expanding on the solution as well. The more questions we ask, the better we know about what we need to do and that makes the coding much easier.

I will summarize all the analysis below.

1. Bakitup will allow user to set the source folders to be backed up.
2. Every time Bakitup runs, it will use the source folders as set by user
3. Bakitup allows user to set the destination drive where the files will be copied.
4. Bakitup will notify the user if external drive is not connected.
5. User can set the periodicity of the backup runs. This should be like every X days, every week.
6. Bakitup should handle all scenarios involving abrupt stopping due to user logging logging out of PC or sleeping it or shutting it down.
     6.1 It should track files copied between source and destination.
     6.2 It should maintain the integrity of the files copied.
     6.3 It should continue from the point where it left in the 
         next backup.
7. Bakitup should only do incremental backups whenever it runs.
8. Full backup can be triggered by the user, if they need it.

The above states more clearly what the software should do. Though there may be many others, we can start with this list.

What you see as one line stating ‘I have a pain of backing up my folders on my PC onto a external drive’ is normally termed as a business requirement. And all that we expanded from that above are the technical requirements which drive the software construction or the coding part.

In the post-3 we will write code in python for the above list of requirements.