Home Schedule Reading Projects People
Mac Logo Windows Logo Minix Logo Sun Solaris Logo Linux Logo

Project 4: Scheduler - QUFiX

I am planning on making a few minor refinements, but only clarifications and hints.

Assignment Day September 26, 2010 (Wednesday - midnight)
Due Date October 14, 2010 (Thursday 5 PM, but no penalty until the start of October 15). Note this project has an overlap with the synchronization project, so make sure you schedule your time appropriately.

Collaboration Policy - Read Carefully

You can work on this project with a partner, and you are encouraged to discuss the assignment with other students or teams on the email list. You may consult (but not copy) any outside resources you including books, papers, web sites and people (but no penguins or sea urchins).

Note if you work on a team you are required to include a statement to specify the contributions of each team member. The statement can be up to several sentences long, describing the tasks of each individual [see examples here of contribution statements here - note this is in the context of a research article but it still a real world example (starting at 2nd paragraph) here ].

If you work on your own - you will get a 5% bonus provided that your base grade is above 60%). As this will require more effort.

If you use resources other than the class materials, indicate what you used along with your answer.

You have probably guessed it by now - every project will have a similar paragraph to the one above but with slight variations, of-course.

 

Lessons to date: 1) Participate on the google MINIX group, this is a fantastic resource, 2) Use google - you won't get penalized - it is a win-win tool 3) Use the class list, but to get a reply - be courteuous and outline what resources you have tried before posting, e.g., I tried searching on google for "add a server to MINIX", I posted on the google MINIX but I did not get a reply 4) MISC - IF you do work on a team - and get stuck - try to use your partner's set-up. 5) backup frequently - your kernel will crash - and it will crash often.

Objective:

The main objectives for this project are:

The main goal for this project is to adda system call (via a tutorial) and to modify the MINIX 3 scheduler to be more flexible. To achieve this goal you must implement:

This tutorial/project is inspired by Darrel Long and used by permission and many others.

Tutorial / References

MINIX resources programming/tutorials:

There are several MINIX resources that I found useful.

http://en.wikipedia.org/wiki/MINIX_3 (see great links at bottom).

http://www.minix3.org/doc/A-312.html (outlines the process nicely)

http://wiki.minix3.org/en/UsersGuide/DoingInstallation (newer and above, but the first listed is still useful).

http://wiki.minix3.org/en/UsersGuide/RunningMinixOnVmware(insights on running MINIX on WMware.

http://www.minix3.org/manpages/index.html (Minix man pages)

How to implement a system call:

http://www.phien.org/ucdavis/ta/ecs150-f03/syscall.html http://www.cis.syr.edu/~wedu/seed/Documentation/Minix3/How_to_add_system_call.pdf
http://fixunix.com/minix/255067-add-system-call.html

VMWare resources programming/tutorials:

http://www.vmware.com/products/player/ (free VMware player)

http://www.vmware.com/products/fusion/ (WMWare for mac and it links to 30-day trial - we are hoping to get/setup a 12 months license for our are waiting for an approval for this - it looks like a possibility right now - please email me if you are interested in this).
If you find other resources please post links on the mailing list for the class.

Coding Strategy:

And it doesn't hurt to mention this again:

As previously mentioned this project will teach you how to experiment with a real operating system kernel(s)- and as a consequence you will learn to work in such a way that deals with a computer crashes (well not quite since we are working in a virtual machine) and / or end up with an OS that doesn't work.

We suggest that you manage multiple kernels, and always have access to at least one working kernel. Back up your work frequently and work in small incremental steps. Incremental steps is very important. A step could be as small as one line of code, or changing the value of an assignement.

Basics:

This center piece of this project is to hack the modify the scheduling policy of MINIX. There are three tasks in the project and TWO of the taks can be completed independently. The three tasks are:

  1. Implement: a system call in MINIX
  2. Implement: 3 simple scheduling policies
  3. Experiment & Evaluate: Compare and analyze the performance of the original and new scheduling policies.
  4. Challenge: Bonus for UG/Some elements are required for Grads - make it so that a system call changes the scheduling policy on the fly (note this can be tricky because of the multiple queueus in the original MINIX). You need to flush the queues.

Part 1: A Simple System Call

You are to write a system call, numberprocs(), that returns the number of processes currently running in the system. Note this part is independent of Part 2, but it will give you a gentle introduction to the OS hacking.

[Link to an example tutorial is below - there are more links in the 'red' box above]

http://www.phien.org/ucdavis/ta/ecs150-f03/syscall.html

Part 2: Scheduling:

Unless explicitly specified, all source code can be found in /usr/src/.

Currently the Minix scheduler maintains 16 queues of processes. System processes use queues 0 to 6. Queue 15 is for the idle process which only runs if there are no other processes to run. The scheduling algorithm finds the highest priority queue that is not empty and picks the process at the head of the queue. Scheduling is round robin for each queue.

In this assignment you will implement other scheduling algorithms and compare them with the current Minix scheduling algorithm. These algorithms are described below:

  1. Modify the Minix scheduler so that you use only one queue for user processes. Denote this queue: USER_Q (see proc.h). Do not change the way that a process is added to the queue.
  2. Modify it according to the 1st policy, use only one queue and denote it by USER_Q. Except when a process is added to the queue always add it to the end of the queue (the priotize if a process runs out of a quantum)
  3. Modify the Minix scheduler such that the process selected from a queue is the process with the least amount of CPU scheduling time.

Please read the background section in 2009 assigment to undertand the MINIX coding structure better. Please note that there are some differences between 3.1.7+ and previous versions of MINIX, these are however not major differences.

Part 3: Experiment & Evaluate:

Compare the new schedulers with the original scheduler. This will be done by running a workload generated by two programs: scanfiles.c and timewaste.c. The program in scanfiles.c is an I/O bound program while timewaste.c is a CPU bound program. Both of these are found on the assignments will be available shortly. Your comparison should be based on the time it takes to complete scanfiles.c. You can use the time command which will give you the real time used by a process. You should average times observed over a series of runs since times will fluctuate (we suggest 3 to 5). There is no need to reboot Minix or log in again. You should run the trails in quick succession. You should include the stastics and your conclusion in your report with your results and explain why you think you got the results you have.

Here is scanfiles.c and here is timewaste.c

Part 4: Scheduling System Call:

Challenge: Bonus for UG/ and some elements described below will be required for graduate stuents - make it so that a system call changes the scheduling policy on the fly (note this can be tricky because of the multiple queueus in the original MINIX). Hint: You will need to flush the queues.

Hints

Last year - students requested do know about interrupt handling in order to understand the life cycle of a process better. I have included the discussion below.

MINIX Interrupt Handling (optional reading to understand the life cycle of a process)

If you are interested in the life cycle of a process and details of the mechanics switching process -- read on. As discussed in class the interrupt handler is responsible for the grunt work of switching and 'restarting' (runnable) ready processes (or removing a preempted process form the CPU). The MINIX code for this (that you should not need need to modify) is in mpx386.s (in src/kernel/) (and is in assembly code). Here is the flow of restarting a process using the interrupt handler. Hope that this will give you a better picture.

  1. CPU is stopped.
  2. The CPU automatically goes into kernel mode with further interrupts disabled.
  3. Values are pushed to registers (e.g., EIP) onto the stack
  4. Hardware detects nature of interrupt via a number called the IRQ
  5. CPU uses IRQ number to index into the interrupt descriptor table
  6. The interrupt is handled accordingly (via IRQ number).
    Here the instruction pointer is automatically set to start the execution of the appropriate interrupt handler code in MINIX, this code is in the assembly code file mpx386.s (src/kernel) and the entry point, for clock interrupt handling for instance is IRQ - 0 (I believe - please double check this detail) at _hwint00: hwint_master(IRQ number) in mpx386.s
  7. _restart in mpx386 is the code that runs the process at
    _next_ptr (or _proc_ptr) that is set in
    pick_proc (kernel/proc.c).
    proc_ptr at end of restart will point to the slot of the process table for the process that runs next (which might be the same process that was interrupted).

 

We assume that you are already familiar with makefiles and debugging techniques from your earlier project in this class or from courses such as CS 1730 or the UNIX tutorials held the first week of class by the department. If not (which is unlikely), this will be a considerably more difficult project because you will have to learn to use these tools as well.

This project doesn't require a lot of coding (typically fewer than 250 lines of code), but does require that you understand how to use MINIX and how to use basic system calls. You're encouraged to go to the class discussion section or talk with the course staff during office hours to get help if you need it.

You should do your design first, before writing your code. To do this, experiment by inserting debugging print statements if it'll help. It may be more fun to just start coding without a design, but it'll also result in spending more time than you need to on the project.

IMPORTANT: As with all of the projects this semester, the key to success is starting early. You can always take a break if you finish early, but it's impossible to complete a 20 hour project in the remaining 12 hours before it's due....

Project groups

You may do this project with a project partner of your choice. However, you can't switch partners after this assignment, so please choose wisely. If you choose to work with a partner (and we encourage it), you both receive the same grade for the project. Note if you work on a team you are required to include a statement in a single file called partner.txt to specify the contributions of each team member. The statement can be up to several sentences long, describing the tasks of each individual [see examples here of contribution statements here - note this is in the context of a research article but it still a real world example (starting at the 2nd paragraph here) ]. Be sure to include the names (first and last) of both partners in the file.

Submitting:

Submitting:

  1. You must include a design document for this project (more on this shortly).
  2. Create a file named mypatch that contains the patch to the Minix kernel to print the requested debug information (as described above). Please note that this patch must cleanly apply to a fresh source tree.
  3. Your files must be in a directory named project4.
  4. Put all the materials needed discussed above in the above directory (including your README.txt file)
  5. Submit via the 'submit' command (while on odin.cs.uga.edu)

    {odin:maria} submit project4 csx730

You will need to demonstrate your projects - we will set up a schedul for this, here you need to prepare a short (talking) script 'convincing' us that you have implemented the project appropriately and correctly.

 

 

Design Document:

Your design document should describe the design of your assignment in enough detail that a knowledgeable programmer could duplicate your work. This includes descriptions of the data structures you use, all non-trivial algorithms and formulas, and a description of each function including its purpose, inputs, outputs, and assumptions it makes about the inputs or outputs. A sample design document is available here.

Grading Criteria:

150 total (two categories each worth 60 pts & 90 pts see below), grade will be converted to a 100% grade (so divide by 150).

150
Total		      
60 40 10

Design, structure and pseudo code: 10 system call, & 10 each of the 3 algorithms
10
10
10
20 5

As proof of concept demonstrates that “something” for each component compiles and runs (system call, each components have somthing that runs/compile). This is not expected to be the full fledge system

5
5
5
90 30 10

Implementation and how well it is implemented (more details here from above criteria), where:
30 point is for the system call (core functionality runs, runs runs well and evaluated well), 30 scheduling for first two 1 queue algorithms (core functionality, runs well and is evaluated well), and finally 30 points for the CPU burst strategy (core functionality runs, runs runs well and evaluated well)
10
10
30 10
10
10
30 10
10
10