Project Goals
- Digitizing current written test from end to end
- Digitizing student activities recording
- Reducing paper in printing the test and the student activity log sheet
Option Analysis and Feasibility Study
There are 4 options in project implementation.
First option is purchase existing product. Software procurement will cost a lot. Although some assessment systems are free to use, cost still exist because compulsory component may not be included. For instance, we need to set up our own database. Also, the flexibility is low because existing software may not fully satisfy user needs and modification may impossible.For example, Google forms module is not editable.Difficulty and time required in this option are low since main system is available already.
Second choice is building the system on our own with existing frameworks. This demands lowest level of cost because many frameworks are free to use. Although coding is easier, other difficulties still exist. For instance, choose feasible framework and understand how to use it is not simple. This development approach also consumes more time than full outsourcing. Furthermore, flexibility is not highest because some functions may be restricted by framework used.
Develop whole system on our own will be third option. Cost is higher when comparing to framework aided approach. It requires more manpower. Difficulty is highest because all technical issue will be solved by our own. Lots of time required as well. However, this offers highest flexibility since whole system is designed base on user’s requirement.
Last option is outsourcing thus new system will be developed by vendor. Like purchasing existing software, this may cost user a lot. Such option has high flexibility. The reason is vendor will also design and develop assessment system according user’s expectation. Also, this option has lowest difficulty since related risks and technical issues are transferred to vendor. However, project duration may not be shortened. Procurement steps such as writing Request for Proposal are required. After choosing vendor, we need to give reasonable time to vendor for project development according to budget. Also, we should reserve time buffer in case vendor completed the task late.
Analysis is summarized in below table.
| Options | Vendor existing solution (1) |
Develop own system with existing library / framework (2) |
Develop whole system on our own (3) |
New system developed by vendor (4) |
|---|---|---|---|---|
| Cost | High | Low | Mid | High |
| Flexibility | Low | Mid | High | High |
| Difficulty | Low | Mid | High | Low |
| Time | Low | Mid | High | High |
To obtain the feasible option, we also performed quantitative analysis with this scoring matrix. Positive factors give higher score.
| High | Middle | Low | |
|---|---|---|---|
| Cost | 1 | 2 | 3 |
| Flexibility | 3 | 2 | 1 |
| Difficulty | 1 | 2 | 3 |
| Time | 1 | 2 | 3 |
Quantitative analysis result is shown in following table:
| Options | Vendor existing solution (1) |
Develop own system with existing library / framework (2) |
Develop whole system on our own (3) |
New system developed by vendor (4) |
|---|---|---|---|---|
| Cost | 1 | 3 | 2 | 1 |
| Flexibility | 1 | 2 | 3 | 3 |
| Difficulty | 3 | 2 | 1 | 3 |
| Time | 3 | 2 | 1 | 1 |
| Total | 8 | 9 | 7 | 8 |
After considering above criteria, option 2 was chosen because it is cheap and has the flexibility to implement user specific requirement. The time required can also fit the project schedule.
Project Approach
We will adopt a waterfall approach in the project which user requirements will be clarified before starting development. Project scope must be defined before development starts and the change in scope will be carefully managed by the project manager.
Project Scope and Requirement Prioritization
Here are expectations from users in this project.
- Assist test paper preparation based on question bank
There will be a question bank in system and leaders will design question paper from questions in the bank. - Member can do the test with their own devices.
The assessment system should be able to work on different platform. - Provide test paper printing function
Test paper set in system can be printed out for any contingency. For instance, if the network is down, test can conduct on time with hard copies. - Arbitrary order of questions
Question order in same test paper among students should be different. - Mark answers automatically
System should able to perform test grading automatically. For typing questions, grading module should able to detect and omit minor typo mistakes. - Leaders can edit marking of the system
After system marking, users should be able to update the test result. - Record members’ other accomplishment records
Achievements of members, including test result and qualifications, should be centralized in this system. - User-friendly interface
No special training required before using this system. For members, - Support different types of questions
Multiple choices and fill in the blanks as well as short questions should be included in system. Besides, new type of questions designed by users should be accepted. - Centralize teaching resources
User states that no centralized platform for teaching resource sharing.
| Requirement | Rationale | |
|---|---|---|
| Core Functions | ||
| 1 | Assist test paper preparation based on question bank |
This is the most important function of the assessment system |
| 4 | Arbitrary orderof questions | Simple function implementation and able to increase satisfaction |
| 5 | Mark answers automatically | This feature is valuable because it saves lot of leader manpower on test paper marking |
| 6 | Leaders can edit marking of the system |
Edit marking is another core function due to automatic marking is included |
| 8 | User-friendly interface | Simple interface must be provided. Different types of users should able to use this system without special training |
| Lower Priority Core Functions | ||
| 2 | Member can do the test with their own devices |
Cross platform is time consuming in both coding and testing. |
| 3 | Provide test paper printing function |
Uncertainties exist when constructing printing function with various printers and encoding. |
| 9 | Support different types of questions |
Flexibility on question type is important but may require large amount of resource. Special design required to deal with unseen question types |
| Non-core Functions | ||
| 7 | Record members’ other accomplishment records |
Centralizing all members’ achievements is not main purpose of assessment system. So, this will be constructed when extra resource available and core functions are completed. |
| Excluded functions | ||
| 10 | Centralize teaching resources | This function is excluded because not closely related to assessment system and may be time-consuming.Different from centralization of members’ achievements, this feature is more complicated due to teaching material are in different forms |
Besides above expectations from user, we propose following extra functions:
- Test result report and analysis
We purpose an assessment result analysis. Various kind of analysis is available after test result digitalization. Analysis on question with lowest correct rate and member performance are examples. This feature helps users in question paper design. - Question frequency detection
User concerned about frequent use of same questions in different course assessments. Thus, question frequency detection function is suggested. There will be a threshold of question appearance maintained by user. This module will count question occurrence after selection from bank. In case appearance count exceeds threshold, system will show reminder. - Automatic question refreshment
Sometimes question may be unclear or contains typo mistake. We propose a function that question displayed in member’s devices will be updated at once after leader’s modification. Therefore, leaders can update questions during test.