This blog post is a bit of a rant because I am always running across “documentation” that is patently false.
Let me start with some word and phrase definitions. That way we are all on the same page for the discussion.
Document: verb
past tense: documented; past participle: documented.
record (something) in written, photographic, or other forms, supported with or accompanying documentation.
Well: adverb 1. in a good or satisfactory way. 2. in a thorough manner.
“Well documented” versus “Documented well”
When I search the term “documented well” on several different browsers and search engines, it always defaults to “well documented” so I have to assume that “documented well” is just a variant of “well documented”. This means the adverb “well” modifies and qualifies the verb, “documented”, expressing a relation of degree in the past tense.
The following is what I feel best describes the phrase "well documented".
"well documented" means a record in written, photographic, or other forms that is supported thoroughly with sufficient relevant documentation to enable understanding of the facts and the reasons for those determinations. This documentation should include all source data, clearly detailed methodologies, calculations, results, and all criteria be explained plainly.
So now to my views on documentation. In the last few years, I have taken up writing. I never thought that I would, it came from my interest in “Historical Jewelry Manufacturing processes”. I had to learn a lot about proper documentation by reading and studying others’ writing. By that, I mean trying to decipher just what in the hell the authors were trying to say and how they interpreted the references they cited.
It soon became apparent to me that most writers were all too willing to blindly accept what others wrote with no investigation into chasing down primary sources.
It was also apparent that many writers would take very scant and obscure data points and draw wildly broad conclusions from them. Many times I have chased down secondary and tertiary references to understand the thinking of an author.
While 1 plus 2 equals 3 to most of us, many times an author will add 1+2 and somehow get 7 as the answer. then make the highly speculative jump to “7 ÷ 2(2+2) = buttered popcorn”.
Leaving me to wonder just WTF the author was thinking.
So this leads me to be very cautious about any footnote, endnote, or reference that I can not chase down the primary resource for. Because of this, I do not believe in using one bit of evidence and trying to extrapolate “facts” from it. Taking the posture that this type of poor writing triggered me to be angry at this low quality of writing, and not wanting to be seen in the same light, I know that I tend to require more documentation than others in my writing and research.
Has this skewed my outlook? Probably. But not in a way that can be misconstrued by those using my research. And then I discuss what is considered “common knowledge” as that affects the conversation depending on the subject.
Below is how I look at all documentation for the work that I do and the standard I hold others to.
Given: there is no body of evidence for an object, process, or way of thought.
That being said, the following is true in my mind:
- 1 piece of documented evidence:
means that this item, process, or thought is most likely an anomaly and when graphed with other evidence is statistically, a data point that is an outlier that differs significantly from other observations. - 2 pieces of documented evidence:
This probably means that this item, process, or thought is still most likely an anomaly. Do not fall for the trap of forming a hypothesis with this data. - 3 pieces of documented evidence:
This means that this item, process, or thought may not be an anomaly and could, with caution, be part of the main body of evidence on a subject. Be prepared to change or drop your hypothesis if additional evidence becomes available. - 4 pieces of documented evidence:
This is the first significant number of documented instances in my mind. With 4 documentable instances, this item, process, or thought is most likely not an anomaly. It can be cautiously considered as a variant, evolution, or artistic/regional/national indicator or trend. At this point, these 4 instances may be cautiously used to form a hypothesis.
MIT has a good webpage about this and I generally agree with what it says. Please review it at this link: https://integrity.mit.edu/handbook/citing-your-sources/what-common-knowledge.
Below are some excerpts from that webpage and my opinions on them.
What is Common Knowledge?
You may have heard people say that you do not have to cite your source when the information you include is “common knowledge.” But what is common knowledge?Broadly speaking, common knowledge refers to information that the average, educated reader would accept as reliable without having to look it up. This includes:
Information that most people know, such as that water freezes at 32 degrees Fahrenheit or that Barack Obama was the first American of mixed race to be elected president.
Information shared by a cultural or national group, such as the names of famous heroes or events in the nation’s history that are remembered and celebrated.
Knowledge shared by members of a certain field, such as the fact that the necessary condition for diffraction of radiation of wavelength from a crystalline solid is given by Bragg’s law.
However, what may be common knowledge in one culture, nation, academic discipline or peer group may not be common knowledge in another.
In my writing, I try to write aimed at two audiences interested in jewelry. For those who are not educated in jewelry so I try to use simple terms and examples to understand the information I am trying to convey, and for those who are well-educated in the jewelry trade where I am using terms that are not common outside of the trade.
An example would be talking to the client who I did this sketch for and talking to the jeweler who would be making this. (Click on the thumbnail to see it full-sized.)
This ring was a created white-based opal with lots of reds and pink flashes in it. the small gems were pink tourmalines set into platinum. The metal was 18 kt yellow gold with platinum highlights.
To the client, I said, “the platinum was on the small part of the curvy part and goes halfway down the sides as this matches the settings of the gemstones and ties everything together to look nice and to accent the length of your fingers”.
To the jeweler, the notes were “18kt W/ 950 plat tube settings for the Tourms with 950 plat accents on the leading edges of the curvilinear design going down the shoulders of the ring”.
See the difference? What was common knowledge for one audience is not for the other.
So how does this tie into my take on documentation?
In my opinion, a reference should be written for the most educated people who you expect will be reading and evaluating your work. But it should be clear enough and complete enough so that if someone does not understand your reference for any reason, they should be able to use that reference to go and find information that helps them understand what you mean.
I always ask myself:
- Who is my audience?
- What can I assume they already know?
- Will I be asked where I obtained my information?
I also ask myself these questions when I write.
- What do I know?
- What do I think?
- Why do I think that?
When I can answer these questions, I am reasonably sure that I can provide an understandable reference.
And I never accept common knowledge as accurate. (Look up aglet-babies for this.) Just because something is accepted as common knowledge because it has been that way for centuries, does not mean it is so. ALWAYS check every fact.
I know my standards are a bit high compared to others as a reaction to not having graduated high school. I know this leads me to over-document everything as I do not want to be seen as that “uneducated guy”. I know that is a fault of mine but it may be okay. I would rather over-document something than not.
I also asked on FaceBook for opinions with the following post:
So a question for the hive mind.
when someone says an item, a bit of data, or thought is “well documented” how many points of evidence do you need to agree?
I need 4 distinct and relatable points of evidence before I agree something is well documented but I know that I am fairly strict on what I call documentation and the standards that I use.
What are your opinions?
I was surprised by the depth of thought in the comments.
To summarize, I think the answers depended a lot on the field of study most people were in. The amount of evidence required varied quite a lot from mathematics, science, the humanities, arts, and history.
One comment that kept showing up was the need for reliable and credible sources. I saw that peer review was not considered that important as long as the sources were credible. There was no one comment or way of thought that was obviously more important. The field of study made a huge difference in the answers.
The only thing that did stick out in the comments to me was a distrust of too narrow of a view from multiple sources causing a case of confirmation bias. It seems that documentation should encompass as wide of a field as possible.
of course, there will be the occasional outlier statistically but as a general rule of thumb, more documentation, from various sources is better.
