Month: March 2013

Reflection on Evaluation – Week Three – Crit101

Originally published here.

This week we learned how to evaluate the validity and the reliability of research.  After learning about internal and external validity and about determining reliability, we were asked to hone our new knowledge by applying it to the research that we did as a group.  We then were asked to write up a paper than explains our evaluation process.  

There was a lot of learning for this week.  To be honest, I did not conduct very much research in college.  It seems to me to be a missing piece in my college education.  I don't have a Master's Degree so I never wrote or presented a dissertation.  I am still surprised that I did not have a research project that demanded such close consideration and scrutiny in college.  

Research is challenging and time consuming.  Evaluation can really be shocking especially while noticing the flaws in the work you have conducted.  Developing critical thinking is so valuable because it can not only shed light on the work of others, it can help you by preventing weakness in research in the first place.  

These exercises have been very engaging for me.  This course has been a great challenge.  I am proud of the work that I have done so far and pleased with the learning.  I am enjoying using the tools for publishing and communication to great effect.  It is nice to have a purpose when learning.  So far I have expanded my knowledge in the use of Twitter, Google Docs, HTML, Markdown, the Web, YouTube,  Open Badges, and Open Courses. 

As a teacher, this has been valuable for me because I am crafting a course in online literacy for next year.  I have bookmarked many of the resources that we have used in this course to apply to my course next year.  Thanks to the "Open"nature of this class, this is not only allowed but encouraged. 

I am finding a great deal of value in this course.

 

Tags: 

Evaluation Work on Research Project

Originally published here.

Evaluation of the Validity and Reliability of Our Research

Although the research that we did was a great exercise and I learned a tremendous amount through the activity, it is clear that the work we did is dubious at best if you consider its validity and reliability. It is worth taking another look at the data, our methodology, and the tools that we used to conduct our research so that we can better understand where our work is incomplete and weak and so that we develop analytical skills to better understand the shortcomings of all conducted research. We are in the think of the “critical” part of our critical thinking course.

Our task was to support an argument about what is the best way to cook an egg. Obviously, this topic was designed to both make it easy to make progress and then easy to look back on for evaluation. The argument is so subjective that we were bound to fail.

Although we mutually contributed to our research project, we did not collaborate on our research. I will run through the evaluation procedures to which we were introduced this week in Crit101 on the research that I performed.

We were asked to data from several sources so that we could better triangulate. Our document did include three sources of data but I was only able to provide two. The first problem that I perceive in regard to validity is that I did not have enough varied sources of data. I was not able to “triangulate” in order to establish validity or reliability. I will continue the exercise focusing my sources of data.

I’d like to focus my evaluation on my first data source. This source was a cookbook titled The Joy of Cooking (1). Because this is the most recognized resource on cooking that I know of, I took it as an authoritative text. I was hoping to find a mention of a preferred method for cooking an egg. Pretending that I did find support for an argument (which I didn’t), there are several issues with this data source once evaluated.

Considering a cookbook as a tool, it is not the right tool for our research. The purpose of a cookbook is most often to present information and list procedures for the preparation of food. It is often objective by its nature. I should not have expected to find any qualitative data. Perhaps if we had defined the criteria for “best” beforehand. This source could have been used for quantitative data.

I do not know if The Joy of Cooking is a universally recognized authority. If I had food data to support the argument, I would not be able to assume that cooks outside of the United States would recognize its authority. America cuisine is not held in the same esteem as say French cooking. Even in the U. S. there are also thousands of cookbooks. It would be impossible to argue that one cookbook has more authority than the thousands of other resources. I am sure there is a name for a data source that is so watered down with the overabundance of other data. Maybe it would be that this source is over generalized.

The reliability of this source is dubious. When considering cooking, we are literally considering tastes. Tastes are so subjective that this data source is unreliable. Some people prefer spicy food and own a reference, Mexican, Indian, and Jamaican cookbooks. Cooking situations also differ. There are cookbooks that are for camping situations and these sources become more reliable is camping situations.

It is interesting to consider how my “go to” source of data holds up so poorly when you evaluate its validity and reliability.

Reference

Rombauer, Irma S. and Rombauer-Becker, Marioin, 1997. The Joy of Cooking. New York: Penguin Putnam Inc.

Tags: 

Week three: Critique of the #egg research project

Originally published here.

What is validity? Firstly, there are two types of validity we must consider when investigating how valid the results of a study are. There is internal and external validity. Also, what is reliability? Are our results reliable and valid?

According to Graham R. Gibbs (University of Huddersfield), he defines that internal validity is when the evidence found reflects the reality under investigation. As for external validity, he defines it as when the results have relevance beyond the situation being investigated. This can also be know as generalisability. Furthermore he describes reliability, it’s defined as when an investigation is repeated by different researchers using the same methods and the same results are obtained.

We were told to investigate the best way to cook an egg, however seen as ‘best’ could be interpreted in various ways, as a group, we decided to investigate what would be the tastiest method. Therefore seen as we only investigated one line of inquiry around the word 'best’ I think that the internal validity of our results is quite low, because we didn’t investigate the question as a whole so it doesn’t represent the reality under investigation. 

As for external validity, I believe the generalisability is very limited for our results. The survey we conducted via twitter had 18 responses, therefore only a small sample, so we can’t generalise the results to the rest of the population. 

Lastly I need to consider how reliable the actual findings of the investigation are. We did use triangulation, therefore our findings are based on more than one source. However they didn’t all correlate. Smith, Delia (2013) How to fry an egg and Kerrison, Melissa (2013) What’s the best way to cook an egg? [survey] did correlate, but Berry, Mary p84-86, had different results. Therefore our reliability is limited as not all of our sources agreed. In addition, our results have not be tested by any other groups, also reducing their reliability. 

It is always very important to therefore plan an investigation first, and try to consider techniques to improve reliability and validity before you begin the research process. 

Week three: Critique of the #egg research project

Originally published here.

What is validity? Firstly, there are two types of validity we must consider when investigating how valid the results of a study are. There is internal and external validity. Also, what is reliability? Are our results reliable and valid?

According to Graham R. Gibbs (University of Huddersfield), he defines that internal validity is when the evidence found reflects the reality under investigation. As for external validity, he defines it as when the results have relevance beyond the situation being investigated. This can also be know as generalisability. Furthermore he describes reliability, it’s defined as when an investigation is repeated by different researchers using the same methods and the same results are obtained.

We were told to investigate the best way to cook an egg, however seen as ‘best’ could be interpreted in various ways, as a group, we decided to investigate what would be the tastiest method. Therefore seen as we only investigated one line of inquiry around the word ‘best’ I think that the internal validity of our results is quite low, because we didn’t investigate the question as a whole so it doesn’t represent the reality under investigation. 

As for external validity, I believe the generalisability is very limited for our results. The survey we conducted via twitter had 18 responses, therefore only a small sample, so we can’t generalise the results to the rest of the population. 

Lastly I need to consider how reliable the actual findings of the investigation are. We did use triangulation, therefore our findings are based on more than one source. However they didn’t all correlate. Smith, Delia (2013) How to fry an egg and Kerrison, Melissa (2013) What’s the best way to cook an egg? [survey] did correlate, but Berry, Mary p84-86, had different results. Therefore our reliability is limited as not all of our sources agreed. In addition, our results have not be tested by any other groups, also reducing their reliability. 

It is always very important to therefore plan an investigation first, and try to consider techniques to improve reliability and validity before you begin the research process. 

Week 2 – Due 24th March 2013

Originally published here.

What did I learn about the research project?

From reading the articles in the reading section of the weeks work I realised that researching is not as generalised and straight forward as I previously thought. In actual fact it is a well structured, detailed process that takes a lot of thought and time. Also, I learnt that there are many different researching techniques that are useful to different aspects of research.

How successful we’re we in completing the project?

Overall, our team was quite successful as we finished it in the assigned time limit. Although, we had a lack of motivation and organisation and ended up cramming it all in at the last minute. We discussed our ideas well in the discussion section but when it are to actually doing it there wasn’t much communication. To conclude, I think our final result was good although we had a lack of team work and communication.

Here is a link to our final project: https://docs.google.com/document/d/14JenDif7mk0jRWXDp4bp8AIJxuIIKPqgqFnu9viyxHc/mobilebasic

Was I an effective collaborator?

Honestly, I believe that I was an effective collaborator as I was the member of the group that wrote up 90% of the final piece. Also, I made an effort to do the primary research. Although, I could have been more prepared and helped to motivate the group and start the discussion earlier on in the week instead of waiting for someone else to start the discussion.