Tag: assessments

28 Jul 2023

What are inclusive assessments in eLearning?

Assessments form the basis for eLearning. Without them, it can’t be decided whether the learners are ready for work or not. But the assessments should be inclusive which means that nobody is biased against them due to personal problems, or lack of knowledge of local languages. There could also be other disabilities like dyslexia which could prevent a person from attempting a paper-based test and hence it should be taken care of.

Therefore, it’s the job of the L&D team of the company to design inclusive assessments.

Now what should be the characteristics of such assessments:

  • Prepare tests in all formats and common languages:

The test should allow participants to provide oral answers to questions. This makes sure that participants with reading or speaking disabilities are also able to pass the exam with flying colors. Also, the test questions should be translated into different languages so that all the participants can answer them. If a participant has a writing disability like a fractured arm, he should be provided an aide to complete the test for him or he can be provided speech-to-text facility for the test.

In fact, visually impaired participants should also be given the same facility so that they have no problems doing the test. The test could be converted into the braille format for them and they must have a speech-to-text typewriter so that they can complete the test.

The test takers should not have problems attempting the questions due to the language. The language of the questions should be simple and comprehensible for all. Apart from that, the questions should not be specific to a particular region. For example, if the questions are being attempted by someone from a specific country, then the questions should keep in mind his cultural norms.

There should not be phrases used in such questions which such test takers are not familiar with.

  • Relevant questions:

The aim of any good test is to the knowledge of someone who is going to work in a certain position in the company, whether he is apt for the role or not. The test creator must see to it that someone is not made to answer irrelevant questions in the test which are not related to his caliber to perform the job duties. A digital test is the norm these days since it can be easily evaluated by software and the test results are generated quickly. But someone, who is not required to type so much as a part of his/her role must not be given a test with a lot of typing requirements because it’s useless to check this skill.

Even if a candidate scores well on this test, it does not measure his ability to do good at his job which does not require typing. Therefore, a test must not suffer from construct irrelevant variance which means including impertinent questions in the test which don’t indicate anything in the context of certain skills.

There are also engineering test for employees to check whether they can fix machines in the future. But this test must not include questions in the local language containing intricate words which can’t be understood by someone who is not a fluent speaker. He is required to fix machines but not required needed to speak a language coherently.

Therefore, a test checking the engineering capabilities should be translated into foreign languages too for foreign engineers to attempt it. This is what the test creator has to keep in mind.

How to formulate test questions:

What is the construct of a test?

Whatever quality a test seeks to measure in a test-taker is its construct. Constructs are very important to be established before the test so that it does not stray away from its objective.

Hence, it should be the target of the L&D department to see that test content should be as per its construct. If a test content varies from the construct, which means it starts evaluating something other than the construct and this phenomenon is known as the construct irrelevant variance. For example, when a test requires engineering candidates to show an exemplary typing speed, it is varying from its objective of measuring engineering skills.

Then, as per these test results, the scores of the candidates will reflect their typing skills and not their engineering caliber.

Therefore, before designing any test it’s crucial that some steps are followed so that the test results reflect its purpose:

  • Definition of the Construct:

The first step is that the construct of the test should be established clearly.

  • Coming up with questions:

Designing the questions based on the construct of the test.

  • Testing:

Checking the test’s validity is important so that if there are any problems they can be identified before the test is given to the real candidates. It’s better that before a test is given to a sample, it’s chosen carefully. Such a sample should consist of diverse candidates so that the final test is as inclusive as possible.

For example, there can be the inclusion of someone who does not speak the local language in this sample so that the use of any complex term in the local language is detected 

  • Modification:

Revise the test based on the feedback of the sample of the candidates chosen in the previous step.

Test anxiety is also a problem in which a test taker can have trouble answering questions because of the pressure to complete the test within a certain time limit. It’s better that the test is designed in mind, keeping the construct. If the construct is to check the competence of a test taker, there should be a liberal speed limit.

24 Jun 2022

How to reduce bias in leadership assessments?

The leaders sow the seeds of change. But sometimes, leaders cannot carry out their job in the right direction. Organizations must establish a clear strategy for themselves which is vital. It is then the job of the leaders to ensure that the employees adhere to the changes incorporated as part of such strategic implementation and new policies.

Hence the senior management should show some required behavior, including accommodating new changes so that the leaders can make the rest of the team follow in the same direction. When it’s not so, and not-so-clear strategic goals are established, or the top management does not work in collaboration with each other, the middle management or the newly trained leaders can’t deliver the results expected from them.

There can also be problems in communication between the middle and the top management, which prevent an explicit bottom-up interaction as to what problems are faced by leaders. The senior management is only used to having a top-down approach to communication where interaction is only one-sided. There could be infrastructural problems due to which employees can’t work towards the company’s new goals or lack of coordination between different branches leading to an effect on the performance of employees.

It’s true that choosing the wrong leader can occur. It often happens because organizations consider a person’s personality and academic qualifications. The person might have excelled in the past as a leader in a different industry, but he may not be able to adjust to the same role in another industry.

Importance of assessments: 

Corporate learning also requires assessments to measure what the employees have learned. But evaluation should be included in the training situations so that trainers know what is lacking. When assessments are a part of the training, the trainers get to know the reason for lackluster training data, like high time which is taken to complete a lesson.

Leadership assessments also help an organization evaluate whether a chosen candidate is ideal for the role.

These assessments are used in all organizations to assess whether a chosen candidate is good enough to be a leader.

Leadership surveys

To determine such suitability, surveys are given to the employees to determine whether they consider a certain candidate to be the perfect leader. Such surveys contain the written opinions of the employees, ensuring that they can’t deny it later. These are 360-degree surveys also measuring the opinions of the candidates for leadership roles about junior employees.

But there is bias in such survey results

But leadership surveys are not genuine indicators of how good a leader will be. They only give good ratings to someone thought to be a good leader but not an actual good leader. For example, in such surveys, white men are given better ratings and opinions than black men. Even women are not perceived to be better leaders than men.

Therefore, the questions must be carefully curated in a leadership assessment. This way, such assessments would have no scope for racial or gender bias. So, questions have to be framed, asking the employees whether their preferred leader displays a specific behavior.

You must refrain from asking questions in such a survey about whether the leader shows acceptable behavior or not because something unacceptable for someone may not be acceptable to others. Any favoritism towards them will draw positive comments from the participants in such surveys, and hence questions have to be drafted judiciously to get rational answers.

There has been confusion worldwide over the correct leadership behaviors which encourage others to take action. Whether an employee is capable of becoming a successful leader can be judged by the similarity of his behavior to that of successful leaders.

Therefore, the significant patterns of their behavior can be described as important for someone to be an effective leader.

When organizations understand that someone who is not considered a leader but has all the leadership qualities can be promoted to this position, it’s beneficial. It’s because when he/she is promoted and shows desirable behavior as a leader, the others can all learn from his example. The employees can be encouraged to select the correct leaders through such steps:

  • Ask employees to rate a leader in comparison to the best leader:

People must be shown the right examples of a leader before taking part in such surveys, i.e., assessments. First, they can be asked to evaluate how effective a model leader is to them. Then they can be made to evaluate a real leader. When the employees have an image of an excellent leader, they are likely to rate a candidate leader in comparison with the former rather than rating the latter based on his personality traits such as his complexion, etc.

So, the raters in such surveys should be questioned about the essential leadership attributes. When they answer such questions, the inherent bias in mind towards someone that he is great leader material is removed. Hence it’s important to provide an example of a quintessential leader to company employees.

  • Ask employees to quote specific examples:

The second way is to ensure that the raters are giving the correct reason behind selecting a leader in such assessments. They should be asked why they consider a certain leader great. Has he shown some exemplary behavior because when they are asked to come up with behavioral examples and can’t, they know that a certain person does not qualify to be a leader?

For example, they might have given a high score to a potential leader on a certain behavioral parameter. Still, when raters can’t recall any incidents of such behavior, they realize they don’t have a real reason to select that leader. After being unable to come up with any incident reflecting such behavior, the raters can adjust their scores given to leaders.

When the leaders see such scores, they can use such feedback to rectify their behavior towards others, making it more acceptable as a leader-like behavior.

  • Make sure people take a reasonable time to rate prospective leaders:

People also don’t have enough time to evaluate whether a person has the skills to be a leader. Therefore, people use heuristics while evaluating a leader due to a lack of time. But raters should be informed that they are not pragmatically assessing the information related to the leaders. Hence the raters should know that unnecessarily criticizing a leader can lead to negative consequences for him.

X (Twitter)