Evaluation Work on Research Project

Originally published here.

Evaluation of the Validity and Reliability of Our Research

Although the research that we did was a great exercise and I learned a tremendous amount through the activity, it is clear that the work we did is dubious at best if you consider its validity and reliability. It is worth taking another look at the data, our methodology, and the tools that we used to conduct our research so that we can better understand where our work is incomplete and weak and so that we develop analytical skills to better understand the shortcomings of all conducted research. We are in the think of the “critical” part of our critical thinking course.

Our task was to support an argument about what is the best way to cook an egg. Obviously, this topic was designed to both make it easy to make progress and then easy to look back on for evaluation. The argument is so subjective that we were bound to fail.

Although we mutually contributed to our research project, we did not collaborate on our research. I will run through the evaluation procedures to which we were introduced this week in Crit101 on the research that I performed.

We were asked to data from several sources so that we could better triangulate. Our document did include three sources of data but I was only able to provide two. The first problem that I perceive in regard to validity is that I did not have enough varied sources of data. I was not able to “triangulate” in order to establish validity or reliability. I will continue the exercise focusing my sources of data.

I’d like to focus my evaluation on my first data source. This source was a cookbook titled The Joy of Cooking (1). Because this is the most recognized resource on cooking that I know of, I took it as an authoritative text. I was hoping to find a mention of a preferred method for cooking an egg. Pretending that I did find support for an argument (which I didn’t), there are several issues with this data source once evaluated.

Considering a cookbook as a tool, it is not the right tool for our research. The purpose of a cookbook is most often to present information and list procedures for the preparation of food. It is often objective by its nature. I should not have expected to find any qualitative data. Perhaps if we had defined the criteria for “best” beforehand. This source could have been used for quantitative data.

I do not know if The Joy of Cooking is a universally recognized authority. If I had food data to support the argument, I would not be able to assume that cooks outside of the United States would recognize its authority. America cuisine is not held in the same esteem as say French cooking. Even in the U. S. there are also thousands of cookbooks. It would be impossible to argue that one cookbook has more authority than the thousands of other resources. I am sure there is a name for a data source that is so watered down with the overabundance of other data. Maybe it would be that this source is over generalized.

The reliability of this source is dubious. When considering cooking, we are literally considering tastes. Tastes are so subjective that this data source is unreliable. Some people prefer spicy food and own a reference, Mexican, Indian, and Jamaican cookbooks. Cooking situations also differ. There are cookbooks that are for camping situations and these sources become more reliable is camping situations.

It is interesting to consider how my “go to” source of data holds up so poorly when you evaluate its validity and reliability.

Reference

Rombauer, Irma S. and Rombauer-Becker, Marioin, 1997. The Joy of Cooking. New York: Penguin Putnam Inc.

Tags: 

Week three: Critique of the #egg research project

Originally published here.

What is validity? Firstly, there are two types of validity we must consider when investigating how valid the results of a study are. There is internal and external validity. Also, what is reliability? Are our results reliable and valid?

According to Graham R. Gibbs (University of Huddersfield), he defines that internal validity is when the evidence found reflects the reality under investigation. As for external validity, he defines it as when the results have relevance beyond the situation being investigated. This can also be know as generalisability. Furthermore he describes reliability, it’s defined as when an investigation is repeated by different researchers using the same methods and the same results are obtained.

We were told to investigate the best way to cook an egg, however seen as ‘best’ could be interpreted in various ways, as a group, we decided to investigate what would be the tastiest method. Therefore seen as we only investigated one line of inquiry around the word 'best’ I think that the internal validity of our results is quite low, because we didn’t investigate the question as a whole so it doesn’t represent the reality under investigation. 

As for external validity, I believe the generalisability is very limited for our results. The survey we conducted via twitter had 18 responses, therefore only a small sample, so we can’t generalise the results to the rest of the population. 

Lastly I need to consider how reliable the actual findings of the investigation are. We did use triangulation, therefore our findings are based on more than one source. However they didn’t all correlate. Smith, Delia (2013) How to fry an egg and Kerrison, Melissa (2013) What’s the best way to cook an egg? [survey] did correlate, but Berry, Mary p84-86, had different results. Therefore our reliability is limited as not all of our sources agreed. In addition, our results have not be tested by any other groups, also reducing their reliability. 

It is always very important to therefore plan an investigation first, and try to consider techniques to improve reliability and validity before you begin the research process. 

Week three: Critique of the #egg research project

Originally published here.

What is validity? Firstly, there are two types of validity we must consider when investigating how valid the results of a study are. There is internal and external validity. Also, what is reliability? Are our results reliable and valid?

According to Graham R. Gibbs (University of Huddersfield), he defines that internal validity is when the evidence found reflects the reality under investigation. As for external validity, he defines it as when the results have relevance beyond the situation being investigated. This can also be know as generalisability. Furthermore he describes reliability, it’s defined as when an investigation is repeated by different researchers using the same methods and the same results are obtained.

We were told to investigate the best way to cook an egg, however seen as ‘best’ could be interpreted in various ways, as a group, we decided to investigate what would be the tastiest method. Therefore seen as we only investigated one line of inquiry around the word ‘best’ I think that the internal validity of our results is quite low, because we didn’t investigate the question as a whole so it doesn’t represent the reality under investigation. 

As for external validity, I believe the generalisability is very limited for our results. The survey we conducted via twitter had 18 responses, therefore only a small sample, so we can’t generalise the results to the rest of the population. 

Lastly I need to consider how reliable the actual findings of the investigation are. We did use triangulation, therefore our findings are based on more than one source. However they didn’t all correlate. Smith, Delia (2013) How to fry an egg and Kerrison, Melissa (2013) What’s the best way to cook an egg? [survey] did correlate, but Berry, Mary p84-86, had different results. Therefore our reliability is limited as not all of our sources agreed. In addition, our results have not be tested by any other groups, also reducing their reliability. 

It is always very important to therefore plan an investigation first, and try to consider techniques to improve reliability and validity before you begin the research process. 

Week 2 – Due 24th March 2013

Originally published here.

What did I learn about the research project?

From reading the articles in the reading section of the weeks work I realised that researching is not as generalised and straight forward as I previously thought. In actual fact it is a well structured, detailed process that takes a lot of thought and time. Also, I learnt that there are many different researching techniques that are useful to different aspects of research.

How successful we’re we in completing the project?

Overall, our team was quite successful as we finished it in the assigned time limit. Although, we had a lack of motivation and organisation and ended up cramming it all in at the last minute. We discussed our ideas well in the discussion section but when it are to actually doing it there wasn’t much communication. To conclude, I think our final result was good although we had a lack of team work and communication.

Here is a link to our final project: https://docs.google.com/document/d/14JenDif7mk0jRWXDp4bp8AIJxuIIKPqgqFnu9viyxHc/mobilebasic

Was I an effective collaborator?

Honestly, I believe that I was an effective collaborator as I was the member of the group that wrote up 90% of the final piece. Also, I made an effort to do the primary research. Although, I could have been more prepared and helped to motivate the group and start the discussion earlier on in the week instead of waiting for someone else to start the discussion.


Research and Enquiry

Originally published here.

“What is the best way to cook an egg?” was the question we were asked to tackle and to find a solution to. I realised it was going to be quite a challenge seeing as the question was very broad and open to interpretation and we would have to collaborate our research to find an answer. In terms of the research process, I found it very useful that we had guidelines to help with the inquisition as this made the process much more systematic and easier to approach as well. The research and enquiry was very time consuming and required a lot of thought as you needed to draw from various resources and not just type into google the question and assume that the first answer you see is definitive. However, it has been very useful as it has helped me develop my ability to be critical about sources and also cross referencing information to draw the best quality data which involves both qualitative and quantitative data. Quantitative data gives us better objectivity and the results we assume, would be accurate and therefore it gave us a good basis for our research by providing us with data to work from and which was also relatively easy to obtain. Qualitative data gives us richer and more detailed information to work with, although this is much more subject to opinion and bias and so we had to procede with caution whilst handling this form of data. I believe in order to get the best results you would have to draw conclusions from both types of data (triangulation) which is a new skill I have acquired during this week.
In terms of our success as a group working on this project, I found it difficult to communicate with the other members of the group on many separate occasions regarding which direction our research was going to take. This problem has most likely arisen from the fact the discussions had to take place online and we were not on at the same time to properly debate how to answer this question, and some members of the group failed to make an appearance and contribute to the research. This made it difficult to really make good progress on the project as a group as it took a while for it to really start up. Admittedly it took myself a good part of the week to really get going as it took me a while to figure out how to use google docs in the first place! Nevertheless I believe the project would have been more succesful had it not been for the fact everything was done online and if some members of the group were more co-operative and a the moment it hasn’t been published as I’m waiting for a group consensus to approve.
I think i made an effort in collaborating my ideas with others, however as some members of the group did not attempt to make any changes or respond very much it is hard to assess how affective I was as a collaborator. This probably wasn’t helped by the fact none of us where ever online at the same time. However, I believe i could have given more feedback on other peoples input to the project and engaged in more discussion, as well as starting the project slightly earlier! All in all, I think this week has shown some of the difficulties of working in the groups sometimes but also highlighted how useful it can be to share ideas and information.