1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
188
189
190
191
192
193
194
195
196
197
198
199
200
201
202
203
204
205
206
207
208
209
210
211
212
213
214
215
216
217
218
219
220
221
222
223
224
225
226
227
228
229
230
231
232
233
234
235
236
237
238
239
240
241
242
243
244
245
246
247
248
249
250
251
252
253
254
255
256
257
258
259
260
261
262
263
264
265
266
267
268
269
270
271
272
273
274
275
276
277
278
279
280
281
282
283
284
285
286
287
288
289
290
291
292
293
294
295
296
297
298
299
300
301
302
303
304
305
306
307
308
309
310
311
312
313
314
315
316
317
318
319
320
321
322
323
324
325
326
327
328
329
330
331
332
333
334
335
336
337
338
339
340
341
342
343
344
345
346
347
348
349
350
351
352
353
354
355
356
357
358
359
360
361
362
363
364
365
366
367
368
369
370
371
372
373
374
375
376
377
378
379
380
381
382
383
384
385
386
387
388
389
390
391
392
393
394
395
396
397
398
399
400
401
402
403
404
405
406
407
408
409
410
411
412
413
414
415
416
417
418
419
420
421
422
423
424
425
426
427
428
429
430
431
432
433
434
435
436
437
438
439
440
441
442
443
444
445
446
447
448
449
450
451
452
453
454
455
456
457
458
459
460
461
462
463
464
465
466
467
468
469
470
471
472
473
474
475
476
477
478
479
480
481
482
483
484
485
|
.Dd February 2, 2022
.Dt JORTS 7
.Os "Causal Agency"
.
.Sh NAME
.Nm Introducing Jorts
.Nd june's ports
.
.Sh DESCRIPTION
Alright so I've gone off the deep end,
maybe.
After continual frustration with MacPorts
culminating in not being able to install
.Xr nvi 1
on my work MacBook,
I have just gone ahead
and started my own personal ports tree
for macOS.
After a couple of weeks,
I have 32 ports in my tree
and only two remaining requested ports
installed from MacPorts.
.
.Pp
I set out with a couple ideas in mind:
.Bl -bullet
.It
This will be my own personal ports tree.
It only has to work for me.
Since I'm using it on both
my personal Intel MacBook Pro
still running Catalina
and my work M1 MacBook Pro
running Monterey,
it is at least that portable.
.
.It
It's ok to rely on
system libraries and tools
provided by macOS.
I'm not creating a distro,
so it doesn't need to be totally isolated.
This lets me skip really annoying things
like compiler toolchains.
.
.It
Sources get vendored,
either from release tarballs
or with
.Xr git-subtree 1 .
This allows totally pain-free
local patching,
and boy has this paid off.
I can just do what I need to do
to get the thing to build how I want
and commit it in git like anything else.
.Pp
It also means that the tree itself
is entirely self-contained
and doesn't rely on any external sources
or network access.
Honestly with some old and obscure software
it feels like upstream could disappear at any moment,
so this gives me peace of mind too.
.Pp
Another advantage of vendoring upstream sources
is that all of the code installed on my system
(in
.Pa /usr/local
anyway)
is easily inspected,
much like
.Pa /usr/src
on a BSD.
This can be super useful for debugging
or just for reference.
.
.It
Produce simple package tarballs.
They're just the contents of
.Ev DESTDIR
after a staged install.
They get installed for real
by untarring them in
.Pa / .
They can then be uninstalled
(or upgraded)
by removing the paths contained
in the tarball from the system.
.
.It
Track installed packages with symbolic links
to specific package tarballs.
Keep old tarballs around for rollbacks.
This means I can see what's installed
with plain old
.Xr ls 1 !
.Bd -literal
$ ls */Installed
\&...
libretls/Installed toilet/Installed
mandoc/Installed tree/Installed
$ ls -l toilet/Installed
lrwxr-xr-x 1 root staff 19 17 Jan 21:45 toilet/Installed -> toilet-0.3~1.tar.gz
.Ed
.
.It
Use
.Xr bmake 1 .
It's scrutable.
It also knows how to bootstrap itself
pretty well.
Since
.Xr bmake 1
is itself a port in my tree
that would require
.Xr bmake 1
to build and install,
I wrote a small
.Pa Bootstrap
shell script
to install
.Xr bmake 1
.Dq manually
then use that
.Xr bmake 1
to build and install its own port.
It also requires a bit of care
when upgrading the
.Xr bmake 1
port since macOS
rather doesn't like a binary
deleting itself while it's running.
.
.It
No GNU software.
I simply refuse to do it.
To that end,
prefer configuring/building with
.Xr cmake 1
where at all possible.
I fell into this early on
since I originally just wanted to install
.Xr nvi 1
and
.Sy lichray/nvi2
is a better upstream source these days
that uses
.Xr cmake 1 .
.Pp
With a port and support for
.Xr cmake 1
in
.Pa Port.mk ,
I can make changes to
.Pa CMakeLists.txt
files without issue.
I can also vendor upstreams
directly from git
rather than having to find
release tarballs with generated
.Pa configure
scripts and so on.
When I need to make changes
to the build systems of projects using autotools,
I either have to have autotools installed
(from outside my tree)
or painstakingly reflect my edits by hand
in the generated files,
both of which suck hard.
.El
.
.Pp
Ok so that's actually quite a number of ideas.
But they have come together
into something surprisingly usable
surprisingly quickly!
Like I said,
this is only intended to be
my own personal ports tree,
but I hope that some of these ideas
are interesting
and maybe inspire others
to explore similar approaches.
.
.Pp
But wait,
I'm not done yet!
There are some other interesting things
that I came up with along the way,
and also some complaints
about some upstreams,
but I'll try to keep those to a minimum.
.
.Pp
So it turns out that dependencies are hard.
Who knew?
It's easy enough to enforce
direct dependencies
at build time
by just checking for the required
.Pa Installed
symlinks.
It's less straightforward
to do this recursively,
which you need if
you want to be able to say,
.Do
Install
.Xr nvi
for me!
.Dc
and get
.Xr ncurses 3 ,
.Xr cmake 1
and
.Xr pkgconf 1
installed first
if they aren't already.
.
.Pp
Rather than trying to do all that in
.Xr bmake 1 ,
I wrote a shell script called
.Pa Plan ,
which itself produces a shell script.
Given a list of ports
to install or upgrade,
it recursively gathers their dependencies
and feeds them to
.Xr tsort 1 ,
which is a neat utility
which topologically sorts a graph.
In other words,
it determines the order
in which the graph of dependencies
should be installed.
The
.Pa Plan
script produces a list of
.Xr bmake 1
commands to make that happen
on standard output,
which can be piped to
.Xr sh 1 .
So,
the way to say the above is:
.Bd -literal -offset ident
$ ./Plan -j4 nvi | sh -e
.Ed
.
.Pp
Now,
what's missing from this approach
is the ability to automatically
uninstall no-longer-needed dependencies.
It's something I've criticized Homebrew for lacking
and one of the reasons I started using MacPorts,
so it's somewhat ironic that
my own system lacks it as well.
However,
I don't think it's much of a problem,
since I'm only packaging
what I actually want installed
in the first place.
On my personal computer,
I have all 32 of my ports installed,
and I expect that to continue.
I can always keep using MacPorts
to install things I only intend
to use temporarily.
.
.Pp
Another thing I was slightly concerned about
from the beginning was disk usage.
I think the benefits of vendoring sources
far outweigh the cost in storage,
but it would be nice to at least minimize that cost.
Previously,
I wrote about
.Xr git-sparse-checkout 1 ,
which allows you to only have certain paths
checked out in your git working tree.
Since port sources aren't always interesting
and only
.Em required
while actually building the port,
it makes sense to not have them always checked out.
.
.Pp
Rather than manipulate
.Xr git-sparse-checkout 1
myself,
I added support for it
directly into
.Pa Port.mk .
If sparse checkout is enabled,
building a port will automatically
add its source tree to the checkout list,
and cleaning that port will
remove it from the list.
At rest,
only the port system itself
and the package tarballs
need to be present on the file system.
.
.Pp
It turns out that upstream
build system behaviour
is super inconsistent,
even among projects using
the same tools.
I started collecting a list of checks
to perform on the output of my port builds
to make sure they didn't do anything weird.
They live in
.Pa Check.sh ,
which gets run
when a package tarball is created.
The current list of checks is:
.Bl -bullet
.It
Check for directories not included by
.Ev PACKAGE_DIRS .
In other words,
make sure the port isn't
trying to install anything
outside of
.Pa /usr/local .
Sometimes this makes sense,
though,
which is what
.Ev PACKAGE_DIRS
is for.
.It
Check for references to PWD,
i.e. the build directory.
This can mean the build
didn't understand
.Ev PREFIX
and
.Ev DESTDIR
correctly,
or that it built with debug info.
.It
Check for binaries without manuals.
If your software installs an executable in
.Pa bin
but not a manual page,
your software is incomplete!
Sometimes this just means
I missed an extra documentation install target.
.It
Check for dynamic linking to outside objects.
In other words,
if something ended up linking to
a library installed by MacPorts
rather than the one from
.Nm jorts
or macOS.
.It
Check for dynamic linking
to system libraries
.Nm jorts
provides instead.
Similar to the last one,
if both macOS and
.Nm jorts
provide a library,
check that ports link with the latter.
.It
Check for scripts with outside interpreters.
This is analogous to the linking checks
but for scripts,
checking that their shebang lines
refer to interpreters installed
by macOS or
.Nm jorts .
.El
.
.Pp
A number of my ports
still fail some of these checks,
but I have fixed a lot of problems
the script called out.
.
.Pp
Speaking of problem ports...
git's build system is truly awful.
I'm sorry,
it's just really disappointing.
On the upside though,
I did manage to patch it
to use
.Xr asciidoctor 1
directly to generate manual pages
from asciidoc source,
rather than generating docbook or whatever
then converting that.
One less build dependency!
I also fixed up curl's
.Pa CMakeLists.txt
(which I guess are normally only used on Windows)
to build and install documentation properly.
And I got libcaca's Cocoa driver working again!
Very important to be able to run
.Xr cacafire 1
in a Cocoa window.
.
.Pp
Shout out to SDL2,
which didn't require any patching
or extra options beyond
.Ev USE_CMAKE=yes .
Model upstream.
.
.Pp
Some other odds and ends:
I like being able to name ports how I want
(for example,
.Sy ag )
and use my own port version convention,
using
.Ql +
to append VCS revisions
and
.Ql ~
to append port revisions.
I don't think those are likely
to ever clash with upstream versioning schemes.
Not that I even need to follow upstream versioning.
There is no reason the version number of
.Xr dash 1
should start with a zero.
.
.Pp
Speaking of versions,
a big downside of maintaining your own ports tree
is that you actually need to update it.
Thankfully,
once I packaged
.Xr curl 1
and
.Xr jq 1
(which needs a new release dammit,
it's been 4 years and the build is broken
on macOS),
I could use the Repology API
to check if I'm behind everyone else.
Far more reliable than
trying to automate checking upstreams
for new versions.
That lives in the
.Pa Outdated
shell script.
.
.Pp
Phew!
I wrote a lot about this.
It feels a little self-indulgent,
but I've had fun working on this
and want to share.
If anyone else tries anything similar,
or is weird enough to give
.Nm jorts
a try themselves,
I'd love to hear about it!
.
.Sh SEE ALSO
.Lk https://git.causal.agency/jorts/
.Pp
.Lk https://youtu.be/Sx3ORAO1Y6s
.
.Sh AUTHORS
.An june Aq Mt june@causal.agency
.Pp
Listening to
.Em Arcade Fire \(em Arcade Fire (EP) ,
.Em Arcade Fire \(em The Suburbs .
.Pp
Typed on a brand new
Leopold FC660M
with Cherry MX Red switches.
Lovely keyboard.
|