[mythtv] CMake

[David Hampton]

On Fri, 2024-07-12 at 17:51 -0400, Scott Theisen wrote:
>  On 7/12/24 14:37, David Hampton via mythtv-dev wrote: 
> >  On Fri, 2024-07-12 at 04:02 +0000, Gary Buhrmaster wrote: 
> > >  On Fri, Jul 12, 2024 at 2:14 AM Scott Theisen
> > > <scott.the.elm at gmail.com> wrote: 
> > > > How do you uninstall the installed programs? 
> >  
> > CMake builds don't create an uninstall target, but it would be
> > possible
> > for us to add one.
> > 
> > https://gitlab.kitware.com/cmake/community/-/wikis/FAQ#can-i-do-make-uninstall-with-cmake
> >   
>  I think that is an incredibly stupid design decision on CMake's
> part.  We should definitely have a top level uninstall target that
> uninstalls everything.  I would also like a target to uninstall only
> the plugins (since they are optional).  Uninstalling just MythTV or
> any of the included libraries doesn't make sense to me since then you
> would have to account for the dependencies.

I suspect this would end up with an uninstall target for each of the
sub-projects, with the super-project orchestration calling them in
reverse order.  (The link appears to be only for a single level
project.)  You should be able to call the plugins uninstall target
directly.

> > > > Is there a way to only build, but not install, the plugins?
> > > 
> > 
> > I tried a couple of experiments this morning.  It could be done
> > with an
> > option at the top level, but it wouldn't be the prettiest given
> > where/how the cmake ExternalProject allows substitution.  It seems
> > like
> > the CMakefile would need to provide an alternative ExternalProject
> > that
> > doesn't perform an install, and then build one or the other of
> > those
> > projects based on an option.  To switch from installing to not
> > installing the plugins you would also have to do a complete rebuild
> > from the top level. 
>  
>  I have never used any of the plugins and I don't really care if they
> are installed as long as they don't do anything or can be easily
> uninstalled right after installing.

If you don't want to build the plugins, there's already an option for
that.  You could perform most of your builds with out compiling the
plugins, and then perform a final build with plugins at the end to see
if anything broke.

> for installed plugins.  If that could be disabled or if each plugin
> must be explicitly enabled, I don't think it would matter if they are
> installed.

Mythfrontend looks for plugins and calls their initialization routine.
That puts entries into the menu to call the plugins.  I don't think
anything else happens until you select a plugin supplied menu item.

> > > > Can you build but not install the project?  (e.g. when making
> > > > further changes to the code while running a test with different
> > > > changes.) 
> >  
> > Once you've built/installed the super-project the first time, you
> > can
> > easily compile just the MythTV sub-project.  This is described in
> > the
> > README.CMake.md file.  You can compile just the mythtv directory
> > from
> > the top level:
> > 
> >   $ cmake --build <build-dir>/MythTV-prefix/src/MythTV-build
> >   $ cmake --install <build-dir>/MythTV-prefix/src/MythTV-build
> > 
> > Or you can change into the MythTV build directory and compile from
> > there.
> > 
> >   $ cd <build-dir>/MythTV-prefix/src/MythTV-build
> >   $ cmake --build .
> >   $ cmake --install .
> > 
> > Or if you're working on FFmpeg:
> > 
> >   $ cd <build-dir>/FFmpeg-prefix/src/FFmpeg-build
> >   $ cmake --build .
> >   $ cmake --install . 
>  
> require a rebuild of and possible changes to MythTV and changes in
> MythTV would require a rebuild of and possible changes to the
> plugins.
>  
>  So I'm not sure building only one part really makes sense.

I normally never need to build anything that lives under
mythtv/external.  I'd rather build those libraries once and then reuse
them until the end of days.  Unless, of course, someone else changes
one of those libraries.  If that happens I blow away my install
directory, recompile everything once, and then reuse all the newly
generated libraries for as long as I possibly can.  I *never* want to
compile anything other than the mythtv and mythplugins directories if I
don't absolutely have to.  I think this makes perfect sense.

> > > These (to me) are all related, as the current
> > > CMake recipes do not support what are
> > > considered by some to support the normal
> > > build/install/uninstall workflow.  The
> > > recipes can work for building packages,
> > > but more work seems to be needed to
> > > make this work as developers (and some
> > > users who build from source) would
> > > want/expect. 
> >  
> > The normal 'make' build/install/uninstall workflow.  I think this
> > is a
> > pretty standard 'cmake' workflow.
> > 
> > The current design is that the top level super-project is only an
> > orchestrator, building anywhere from 6 to 32 sub-projects.  Each
> > sub-
> > project is built and installed before the next is started.  I don't
> > know of any way to build the sub-projects without installing them.
> > (I
> > suppose that you could omit the install stage for each sub-project,
> > but
> > then the library/include search paths for every subsequent sub-
> > project
> > would get longer and longer.)  Perhaps there is another way, but I
> > don't think so.  I will readily admit that I am not a cmake expert.
> > 
> > Sub-project designs seem pretty common with cmake builds.  Sub-
> > projects
> > completely isolate the imported libraries from the main projects,
> > guaranteeing API boundaries.  That actually creates an issue
> > because
> > MythTV doesn't respect the FFmpeg API and directly calls several
> > internal functions.  I overcame that API violation by adding an
> > extra
> > step to the FFmpeg sub-project to install several internal header
> > files. 
>  
> am working on removing the last few uses now.  However, there are
> still functions added by MythTV in public headers among other
> changes.

I know you've done a lot of work on FFmpeg and I appreciate it.
  
> > A big part of the reason for my choosing the sub-project design for
> > MythTV is that linux/android/win builds are all done exactly the
> > same
> > way.  The difference between these builds comes down to the number
> > of
> > sub-projects and the arguments to the sub-projects.  It also
> > becomes
> > almost trivial to replace an embedded library with a system
> > library. 
> > And yes, I was thinking of FFmpeg and exiv2 when I chose this
> > design
> > and chose to build those libraries as separate sub-projects instead
> > of
> > as part of the mythtv sub-project.
> > 
> > I do understand that this is different from the current make
> > implementation that builds all of the libraries and MythTV together
> > in
> > one fell swoop, and then installs them.  I think its better in some
> > ways.  For example, with this design you only have to compile and
> > install the mythtv version of FFmpeg once, and all subsequent
> > builds
> > (even complete rebuilds) will treat that installed version of
> > mythtv
> > FFmpeg as a system library.  For everyone other than you and Peter
> > who
> > are actually modifying FFmpeg, I consider that a win.  Its easy
> > enough
> > to force a rebuild of mythtv FFmpeg any time you want.  Just delete
> > the
> > installed mythtv FFmpeg and rebuild the super-project. 
> 
>  A top level build would also build and install a modified FFmpeg,
> replacing the previously installed version, right?

No.  A new top level build will discover the previously installed
mythtv FFmpeg and use that.  You need to delete the previously
installed mythtv FFmpeg if you want to rebuild it.  Everything else is
reused where possible.  The only sub-projects that are always rebuilt
when you recreate a top level build are the mythtv and mythplugins sub-
projects.  (It would be simple enough to add FFmpeg to this list.)  I
love the fact that deleting and re-creating a build directory will
reuse all the previously built libraries and configure itself to only
compile the mythtv bits.

> > The most similar thing to the current 'make' workflow would be to
> > compile the entire super-project once, and then compile/install
> > from
> > the MythTV-build directory as often as you want.
>  
>  I think the most similar would be to have two build targets: one
> that installs to wherever it did before and the other with 
> -DCMAKE_INSTALL_PREFIX=<install_location>
>  set somewhere else (e.g. /tmp or an install folder inside the build
> folder that CMake creates) so that it preserves the API boundaries
> and behaves the same, but won't effect running programs or 

The default 'make' build target is /usr/local.  That doesn't work as a
default build target for a cmake super-project because each sub-project
is built and installed separately.  There is no mechanism in cmake to
invoke sudo for the install stage of each of the sub-projects.  I think
cmake is oblivious to the existence of sudo and raising privileges.

If you want a single build target to compile FFmpeg / libexiv2 /
libmythbluray / libmythdvdnav / libudfread / nv-codec-headers / mythtv
it could be done, but it would require collapsing those six sub-
projects into a single sub-project.  I think that's the wrong approach
and would be moving backwards for MythTV.  Everything under
mythtv/external should be ripped out and replaced by patches against
original sources.  MythTV should no longer ship the source code for any
library that it uses.  It should use system libraries if at all
possible, and if not, it should use package maintainer's source code
with locally applied patches.  This is something that's trivial for
cmake to do, and hard for make to do.  I'm hoping that all your FFmpeg
work will help us get to this point with FFmpeg where it can be ripped
out of the MythTV sources.

>  How does CMake behave when you call
> 
> $ cmake --preset qt5 -DCMAKE_INSTALL_PREFIX=<install_location>
>  more than once?  Create a new differently named folder or override
> the current one?

If you issue that command without first deleting the build tree, cmake
will simply reuse all the cached information in the current build tree.
There's nothing to stop you from having multiple build trees underneath
the same source directory, you just need to specify the build directory
name (the -B option) instead of using the preset's default name.  Some
of my testing involved building all the various permutations, so I had
more than ten different build trees in a single source tree.  These
build trees spanned four cpu architectures, three target operating
systems, and two versions of qt, all from a single source tree.  You
could just as easily have multiple build directories that are all for
linux, differing only in the build arguments.

> Since we use ccache and all of the build artifacts are in CMake's
> build folders, I think this would work and not cause a significant
> delay when switching between the two build targets.

Given ccache support, deleting the install directory and performing a
complete rebuild from the top level doesn't take much more time in a
cmake build than it does in a make build.  Especially once you remove
having to unnecessarily rebuild the libraries in mythtv/external that
never change.  (Yes, I know that *you* need to rebuild FFmpeg every
time, but I consider yours an outlier case.)  Most developers don't
need to build FFmpeg more than once a release.

David