1 Topic by De-Backer (edited by De-Backer 2020-08-28 23:54:47)

Topic: info and discussion about GIT submodules for QElectroTech

i am starting this thread to list the pros cons of GIT submodules.
And what to add and what not to add

what are GIT submodules:
It often happens that while working on one project, you need to use another project from within it. Perhaps it’s a library that a third party developed:
-pugixml
-SingleApplication
-(DXFtoQET)

https://git-scm.com/book/en/v2/Git-Tools-Submodules
https://i.stack.imgur.com/BIBSL.png

but keep all relevant code in 1 repo
why: for bug tracking
see:
source

There are three major disadvantages to one project per repository, the way you've described it above. These are less true if they are truly distinct projects, but from the sounds of it changes to one often require changes to another, which can really exaggerate these problems:

It's harder to discover when bugs were introduced. Tools like git bisect become much more difficult to use when you fracture your repository into sub-repositories. It's possible, it's just not as easy, meaning bug-hunting in times of crisis is that much harder.

Tracking the entire history of a feature is much more difficult. History traversing commands like git log just don't output history as meaningfully with fractured repository structures. You can get some useful output with submodules or subtrees, or through other scriptable methods, but it's just not the same as typing tig --grep=<caseID> or git log --grep=<caseID> and scanning all the commits you care about. Your history becomes harder to understand, which makes it less useful when you really need it.

New developers spend more time learning the Version Control's structure before they can start coding. Every new job requires picking up procedures, but fracturing a project repository means they have to pick up the VC structure in addition the code's architecture. In my experience, this is particularly difficult for developers new to git who come from more traditional, centralized shops that use a single repository.

In the end, it's an opportunity cost calculation. At one former employer, we had our primary application divided into 35 different sub-repositories. On top of them we used a complicated set of scripts to search history, make sure state (i.e. production vs. development branches) was the same across them, and deploy them individually or en masse.

It was just too much; too much for us at least. The management overhead made our features less nimble, made deployments much harder, made teaching new devs take too much time, and by the end of it, we could barely recall why we fractured the repository in the first place. One beautiful spring day, I spent $10 for an afternoon of cluster compute time in EC2. I wove the repos back together with a couple dozen git filter-branch calls. We never looked back.

but keep the repo small source

The reason the team managers finally have accepted the split: the single Git repo (550 MB) was requiring 13 minutes to be cloned on Windows (one minute on Linux).

In essence: submodules for
- a third party library
- a piece of data that cannot negatively affect the code, and one that can become too large
  eg Elements, titleblocks, and maybe docs (fetches the data from the code)

and for large files such as audio samples, videos, datasets, and graphics there is git lfs but let's wait with that. https://git.tuxfamily.org/ does not support this, I think.

cloning:  this requires extra instruction

git

git clone qelectrotech.git qet

VS

git and submodules

git clone --recurse-submodules qelectrotech.git qet

if I am correct, QElectroTech consists of
- project editor
- element editor
- Title block template editor
- (DXFtoQET)
- qet-tb-generator
- ????

programming can quickly get complicated see: https://de-backer.github.io/qelectrotec … ditor.html
for a graphical representation of QETDiagramEditor Class (collaboration diagram).

Obviously git is valuable for finding bugs.
but what if you need to debug on user input
eg: "find class foo that ruined bar x"

I recommend splitting the code but not necessarily making it into extra submodules.

Documenting the code is recommended, but it is easy to forget.
It is more interesting if the code is self-explanatory.
eg https://de-backer.github.io/qelectrotec … indow.html


more info to come....

2 Reply by De-Backer (edited by De-Backer 2020-09-02 21:38:34)

Re: info and discussion about GIT submodules for QElectroTech

draft: convert script

# make submodule qelectrotech-elements of the qelectrotech-source
git clone https://github.com/De-Backer/qelectrotech-source-mirror.git qelectrotech-elements
 
cd qelectrotech-elements
 
# remove all (git and file's) except elements dir
# note! see: https://git-scm.com/docs/git-filter-branch
git filter-branch --prune-empty --subdirectory-filter elements/ -- master
 
git remote add https://github.com/De-Backer/qelectrotech-elements.git
 
git push qelectrotech-elements master
 
cd .

to create a submodule for the elements I'm still in doubt, why: "git filter-branch" is a git history rewrite tool

see: https://github.com/De-Backer/qelectrote … its/master
for the result

git clone https://github.com/De-Backer/qelectrotech-source-mirror.git qelectrotech
cd qelectrotech
 
# add submodule pugixml
git submodule add https://github.com/zeux/pugixml.git
# note:
# qelectrotech.pro
#
#-               sources/pugixml \
#+               pugixml/src \
#
#-           $$files(sources/pugixml/*.hpp) \
#+           $$files(pugixml/src/*.hpp) \
#
#-           $$files(sources/pugixml/*.cpp) \
#+           $$files(pugixml/src/*.cpp) \
 
#! te testen !
rm sources/pugixml/pugiconfig.hpp
rm sources/pugixml/pugixml.cpp
rm sources/pugixml/pugixml.hpp
 
# add submodule SingleApplication
git  rm -r SingleApplication
git submodule add https://github.com/itay-grudev/SingleApplication.git
 
# add submodule qelectrotech-elements
git  rm -r elements
git submodule add https://github.com/De-Backer/qelectrotech-elements.git
 
# init submodules
git submodule init
git submodule update

Re: info and discussion about GIT submodules for QElectroTech

it looks like the KiCAD has started from null. (2017) footprints
they have never used "git filter-branch"

Author: Oliver <oliver.henry.walters@gmail.com>  2017-07-11 02:53:49
Committer: Oliver <oliver.henry.walters@gmail.com>  2017-07-11 02:53:49
Child:  a9b5e170d22074538054b52bacda3a9d2a82d638 (Update README.md)
Branches: master, remotes/server/dev_Heatsink_ALLOYSEED_1144, remotes/server/dev_TO220-3_mod, remotes/server/dev_din_rail, remotes/server/master, remotes/server/working
Follows: 
Precedes: 5.0.0-rc1

    Initial commit

same for the code:

Author: plyatov <Unknown>  2007-05-06 18:03:28
Committer: plyatov <Unknown>  2007-05-06 18:03:28
Child:  822b90f48e6716d78351a0f78b3e048ff9415340 (Initial import of interface translations with russian (not complete) added.)
Branches: master, remotes/origin/4.0, remotes/origin/5.0, remotes/origin/5.1, remotes/origin/drc, remotes/origin/master, remotes/origin/tom-drc-jun13
Follows: 
Precedes: April-2010-release, copy-of-subversion-repo

    Initial import of KiCad.

Re: info and discussion about GIT submodules for QElectroTech

Preferred way to make QElectroTech:

md file

### Cloning

It's important to clone the repository recursively to get all submodules too:

```bash
git clone --recursive https://github.com/qelectrotech/qelectrotech-source-mirror.git qet && cd qet
```

### Updating

When updating the repository, make sure to also update all the submodules
recursively. Otherwise you may get strange compilation errors:

```bash
git submodule update --init --recursive
```

### Building

You can either build QElectroTech using Qt Creator, or you can build on the command
line using qmake. To build a debug version of QElectroTech with qmake/make:

```bash
mkdir build && cd build
qmake -r ../qelectrotech.pro CONFIG+=debug
make -j8
```

For more detailed instructions (including how to set up Qt Creator), see
a www eg=>https://QElectroTech/doc_building.html