[Meeting Wednesday 12th August] Aligning documentation efforts

@MattR It would be fantastic to get things from the assembly/design files. However a lot of us either don’t have the software or the skills to make full CAD assemblies. I think as a minimum if there was a way to get a machine readable BOM from the assembly you could cross reference that with the instruction BOM and get a big red flashing warning if they don’t match. Is it possible to get a BOM out of Solidworks, I remember you saying you had some scripts for this?

@bhaugen This seems really interesting. It would be good to learn more. From a very brief skim of what you posted I think there is a lot of overlap. Though as you say you have a much more general scope. GitBuilding also has the idea of the tree like structure. While we have an nice way to refer to parts and other “steps” we don’t have a good way to track outputs from a step. We have an issue to discus this syntax which I created only a few minutes ago, but follows on from a few discussions.

2 Likes

That’s one of the many benefits of an input-process-output model.

@julianstirling Short answer: yes.

Long answer:

In Solidworks BOMs from an assembly are just a few clicks. My scripts mainly add and extract info to/from the BOM or parts in it. You can add custom attributes to parts. For example I use my own “Type” attribute referring to whether a part is purchased, printed or made. I wrote scripts in the Solidworks API that go through each part in an assembly allowing the user to assign info such as type, tools for adding to an assembly, tools for building, etc. I have also written scripts that add up amounts needed for parts made from common materials and put totals into a Purchasing BOM (think steel profiles and sheet metal). All major subassemblies have their own BOMs that are used for the assembly instructions. There is no way I could keep track of that stuff without the scripts.

Here’s a bit of what happens when I run the documentation scripts: If a part is “Made” the scripts search for a technical drawing, generate a PDF then make a download link that is added to the online BOM. A part that is “Printed” has its STL auto generated and a link to it is also placed in online BOM. Images of all parts are generated and added to the BOM. A tools list for every major sub assembly is gathered and added to the the assembly instructions. Here’s an example of what is produced by the scripts. The only thing I actually wrote by hand is the note at the top and the assembly steps at the bottom of the page.

For OpenSCAD I envision comments being used to attach part info directly to the part, kind of like Javadoc. I’m pretty sure you can make assemblies with OpenSCAD, but if you don’t, you could always use dummy files to document assemblies. For other CAD packages that don’t have a good way to attach documentation info to parts and assemblies, dummy files could also be used though it might be cumbersome.

1 Like

I am glad to hear that there is now more interest in aligning and collaborating. I have been talking about it forever - Let’s “drink our own wine” and act according to what we always preach as the purpose of open source (thrive through collaborative improvement). I don’t see DocuBricks as such an experimental system, it has been used by a fair amount of people over the years incl. several publications in various journals. But the wish list of features is always long and I have more actively taken the current DocuBricks stable-release on again in recent month to modernise it and make it more portable and interoperable. So this discussion is very timely and I would very much like to contribute to such a call so see how we can align initiatives as well as inter-convert our formats.

Are there any example projects already documented with Hardocs? I only learned about it recently and could not find any projects that use it online or in a published project.

Interoperability is important. But I think it is essential that we step back and first look and critically assess our projects for where they don’t meet the communities needs it is only by acknowledging the problems in our own approaches that we will be able to find something fit for purpose.

I say “fit for purpose” not to be rude but to be honest. I think saying that it has been used for a project that is published in a journal is way too low a bar to set. We have documentation in GitBuilding which is referenced in a journal. Being able to document certain simple projects is the first step, we can do that in all of the tools, we can even do that with no special tools using TeX or markdown, or even a word processor. To be fit for purpose as a hardware documentation tool I think at a minimum the software must be

  1. Intuitive to install
  2. Intuitive to use
  3. Easy to navigate the final output
  4. Easy to customise the output
  5. Easy to update documentation as you go

I quite like how GitLab rate features in their own project. Their scale is that a feature is “planned”, “Minimal”, “Viable”, “Complete”, and “Loveable”.

If I was to make a guess for GitBuilding I would say:

  1. Minimal - Requires a command line to install, ok for those comfortable with python
  2. Minimal - I thought it was better OK other people started giving me feedback :frowning_face:. It takes a command line to start it up. The syntax seems to confuse a lot of people even though it is very similar to the syntax we use here in discourse.
  3. Minimal - For a simple project I feel the final output is navigable. But it gets confusing for larger projects
  4. Viable - The structure of the documentation is decided by the user as they make the pages. However certain if you split pages in certain ways it may not work well with the automated bill of materials features.
  5. Minimal - Live editor works well enough that it can be open while you build, You can drag and drop images into it, and the BOM will auto update as you change references in the text. However, we really need a way to connect the editor on the laptop to a phone so you can take photos as you go and they go directly into the docs. You can then push the changes as soon as they are ready into a git repository and automatically build the web page. This works very well for me, but is definitely not optimal for most people. We can also make HTML docs, but people still need a way to get them online

This is my guess for where we are at, but the way they actually score them is to interview users and get a score out of 5. You need to reach 3.14 to be “viable”.

I am not going to score DocuBricks here, but it is worth considering if you agree with the list I have above, and if so where you think you score on it. GitBuilding started because OpenFlexure microscope users and contributors struggled with #2 and #3 in DocuBricks, and we could not fix #2 because of problems with #4. It was for this reason that we moved to pure markdown in the microscope repository. There are things to fix before we move to GitBuilding.

I am not trying to say this to be rude about DocuBricks, but just that I think there will be no coming together if we can’t admit the faults of our own approach. Namely that my approach is so unstructured it is hard to how you should do something, and DocuBricks is so structured that it can be impossible to explain something in the way you want to.

I think the importance of the call is partly for us to have a dialogue, but also for those in the community to give us honest feedback and for us to listen, learn, and adapt. Only by adapting our approach will we be able to build a tool that is “lovable” :heavy_heart_exclamation:

3 Likes

Thank you so much Julian for initiating this conversation. I’m a documentation user, not developer, but we’re doing this exactly sort of thing over in the GOAT community, especially with surveystack.io, farmos.org, landpks, and others. It’s a hard conversation to have, but I think when you do it well it represents one of the most important things that open source communities with personal relationships built on love + trust can do!

Were there a standard that we could also integrate into surveystack.io (which often includes directions-based components in addition to survey-based components) I’d love to consider that as well. I’m also sure that other communities have similar needs (thinking of specifically ROS, OpenAgGPS, FarmHACK, OpenTEAM generally…). If you can clarify a generalized + broadly supported w/ a great team case, I (and I’m sure others) can help pitch this out to communities which could help or contribute.

1 Like

@gbathree Are there any new developments/reports on your and @ryanfobel and @nick’s experience of using hardware documentation tools for your joint production effort? Previously we had discussed this in:

2 Likes

Well, a little backstory first:

Our timing (given COVID) was not great. Nick valiantly tried to move things forward for the first few months, but changes from Ryan’s team and COVID made it unreasonable to take continued risk (in terms of expense and investment) in trying to build a single new entity around a small number (2 core, and 2 in development) products.

In the end, we decided we need to pause and re-evaluate for another day. That being said, I’m definitely still interested in the concept! A few folks are meeting to discuss what else we could do around the original idea or related ideas, and hopefully we can identify a sustainable path forward from there.

So… with regards to documentation, we ended up using mkdocs inside gitlab (so build documentation in a gitlab wiki, produce static page using mkdocs). It’s not really structured as a manufacturing documentation solution, but it was general enough to get us moving forward, it was hosted in the same place as our core code / designs (which is nice) and we assumed that any other solutions in the future would be largely compatible re. using markdown.

It’s not an ideal solution, but it’s not a bad one either. We got some of the documentation in you can see it here (see right hand side for individual pages:

2 Likes

I have made a dudle poll for working out a time for a call.

1 Like

Wednesday the first at 5pm BST seem to be the best time for everyone. Here is a event link to minimise time zone confusions:
https://www.timeanddate.com/worldclock/fixedtime.html?msg=GOSH+documentation+call&iso=20200701T17&p1=2477

Does anyone have a zoom account we can use? If not I will make a jit.si link

See you all there.

1 Like

Due to a slight confusion about the meaning of “meet” (a name that Google has now apparently co-opted) we will do the call at this link:

I agree google are evil. If anyone wants to shout about video platform solutions and the balance of evil vs. open: we have a dedicated thread for that.

I agree that it’s a good discussion to have, and thanks for the initiative to set up a call. I mostly don’t manage to stay on top the many discussion threads in various places due to time limits.

You proposed this list of criteria as a starting point:

It’s hard to make bullet points on this that mean the same to people. For example the first point. I generally agree, but not applicable to most successfully used platforms for documentation, since no installation is needed.
In detail this discussion about usability criteria will be continued in the repository we set up for the purpose, so I’ll stop the detail here.

Also the point easy to customise the output" I don’t think is essential - again those highly used platforms such as thingiverse, instructables, hackster just have a blurb in their format. A structured format removes worries of how to layout and works well for some.

Other points that may be missing are modularity (missing from all major platforms), version control (a very hard problem in Open Hardware IMHO), portability to other formats; ease of identifying the right components and design files; social media functionality to engage, highlight and discuss the content (usually just provided with lots of effort by commercial platforms with money and critical mass). And more.

We had the call THANKS ALL, in particular @gbathree for moderating. We decided to set up a repository on the IEEE open standards platform to discuss about consensus elements and aimf of documentation and see to what degree alignment and compatibility can be achieved.
This is our user group on the IEEE GItLab:

Please register and ask to be a part of the group and discussion there or just submit issues.

3 Likes

Meeting on July 1

@julianstirling @kaspar @MattR

why everyone started?

  • how do I solve my documentation problems?
  • how do a keep broader documentation problems in mind…
  • and how broadly should we consider?

What specific things were missing when you started?

  • (tobey) structured format (XML file) to support flexibility
  • (tobey) simple UI (non command line tool) for starter users
  • (julian/kaspar) bill of materials (vertical integration, flexibility)
  • (julian/kaspar) forked versions (high cost, low cost, etc.)

Is this problem solved now (4 years after starting to discuss it)?

  • (julian) Duzuki open standard is close, but can’t really handle forked versions well (also not open source?), and auto-bill of materials
  • (matt) you need to be able to reformat your inputted data as your project moves forward. Documentation should pull directly from assembly / part files / outputs from actual dev programs.
  • (tobey) at the time of docubricks, the core functionality was achieved - layout and standard format. But feels uptake has been more difficult than expected.

Levels of collaboration

  • same software / stack
  • use common libraries
  • use common metadata
  • conversion from one output (XML) to another
    • oasis open standards, IEEE standards, Dozuki has an open standard tho it’s closed source
    • utility for standard to import / export between different documentation
    • MDITA - markdown and XML conversion back and forth

Who’s developing what, and who wants to (and can continue to?)

  • (julian) currently applying for a grant to work on documentation
  • (tobey) he’s ok to update software to a new platform/service but…
    • the hard-learned UI lessons should be brought along (don’t reinvent that wheel!)
    • the data structure lessons should be brought along too! It should be able to live in github, OSE, Docubricks, etc. etc.
    • He feels that elements should be updated individually / modularity (the word ‘bricks’ helps and use importance in usability!). Thankfully, it seems that those concepts are at least in current version of gitbuilding.

Next steps

  • (tobey) IEEE standard organization, update name + share w/ group
  • (julian) set time for next meeting (once a month)
  • (julian / tobey / kaspar… to forum w/ feedback) outline the ‘vision’ for a structure that learns the lessons of docubricks, integrates the new features we know we need, and is broadly applicable enough to actually achieve some traction.
4 Likes

@Tobey I agree that these things don’t mean the same to all people, but I think that is why we ask lots of people and get a “feel for the room”, both on what questions, how important it is to them, and how they think solutions perform.

“Intuitive to get set up” could replace “Intuitive to install” you don’t have to install to start using Dozuki, but getting a login confused me a lot. Essentially what is the barrier to entry.

Other platforms that exist

highly used platforms such as thingiverse, instructables, hackster

Personally I don’t find these a useful comparison. Their target audience seems to be people who are sharing small hobby projects like “I made a disco ball plant pot from an old mirror and took some photos”. These don’t need revision control, and can probably be adapted into a blurb and some steps.

What about the post doc who is spending 2 years designing and improving an ultra high vaccum x-ray analyser, or a confocal microscope? What about people designing an open source car (not science hardware, but I think we want a tool that is generally hardware applicable if we want uptake). What about all the teams prototyping ventilators during this pandemic? They are not going to get this into an instructables page.

For this type of complicated equipment that takes years to develop, you will need to write and edit and changes and reformat the documentation lots of times. If you wait to the end you will just never write it (never writing it is the norm for PhD/Post-Docs). Integrating this with version control is important for back-up and design history (design history is essential to medical certification!), and is often useful if multiple team members are to work on something together.

As the interest grows in open source medical hardware, people will submit their documentation to regulators who will have strict guidelines. If the documentation output cannot be customised to meet the guidelines then they will have to export and reformat.

If we think about simple projects that only need the feature set of instructables, then why not use instructables? I know it is not open source, but nor is GitHub yet is it widely used in this community. I am more interested in in what professional engineers use. So far, my experience is they prototype first (taking some notes) and then the documentation is created in a word-processing tool with 100s of person-hours. The dream for me is to automate much of the tedious parts so at least a useful skeleton of the documentation can be written while you hare in the workshop/lab pototyping. I want a tool that:

  • McLaren would want to use when prototyping their F1 cars
  • NASA want to use for their next satellite
  • We want to use for the OpenFlexure microscope
  • Our BSc students find easy to use for their first ever hardware project

Structure

The point about a structure to work from can be helpful is a good point. I think a nice way to do this without loosing freedom is for tools to have templates. I can envision adding a new page and having an interface like this:

This gives you structure if you want it but does not constrain you if you need to get down other information.

Modularity

Both of us agree that a modular or hierarchical data-structure is essential to making elements of the documentation reusable for other projects. I have use the IEEE page to lay out a little bit about both of the data structures and then to go into details about how the hierarchy is defined in BuildUp.

1 Like

I have added the next meeting to the top of the thread.

@gbathree @MattR @kaspar @Tobey and everyone else who is interested. This is just a reminder that the call is this wednesday.

Link in the opening post.

Hello everyone, sorry I have been very absent from this forum, this is a very late reply.
Hardocs is a prototype, not more than that. I developed it with 2 more persons to explore solutions to issues I faced when working on hardware design projects in the past.

  • bomake was mostly about “easy” BOM generation, where you could not just make a bom but also write narrative text about the design. Woudlnt it be wonderful if there was something like jupyter notebooks? Maybe it exists, I am not aware.

  • Hardocs: I focused there on making it really easy to write narratitve text, with images, but also structured data with automatic form generation using json schemas.

  • Combatcovid.equipment: Mostly focused on dissemination and search experience, with the case of hardware for covid which was so difficult to find, when people started publishing all kinds of designs. What is interesting about this one, is that it was integrated with github, so it would provide a presentation and search layer on top of github apis.

Some learnings and reflections:

  • There is a lot of great software and cloud platforms, some to limiting or locking, but with nice features to have in an open-source alternative.
  • I have been in general quite naive about the time of developing software, today not so much :wink:
  • Going from prototype to mature stage requires resources and team, or full time commitment, at least in my case. So for me what has stopped me from working more in hardocs, is mostly time.

Regards and apologies for my very asynchronous response