Compare commits

...

33 Commits

Author SHA1 Message Date
Pau Espin 819c17e138 Makefile: run 'bitbake package-index' after bitbake command building packages
It was recently spotted that during a build of 201705-testing where
sysmocom-openvpn-config was being rebuilt, the Packages(.gz) files
(opkg package indexes) were rebuilt before the new .ipk files for
sysmocom-openvpn-config were written to disk. Also then the Packages
file didn't contain a reference to the new sysmocom-openvpn-config
package version. So, it seems there was a race condition where the
package index was built too early.

Yocto documentation provides some hints:
https://docs.yoctoproject.org/dev-manual/packages.html?highlight=bitbake+package+index#build-considerations
"""
Whenever you perform any sort of build step that can potentially generate
a package or modify existing package, it is always a good idea to
re-generate the package index after the build by using the following
command: bitbake package-index
Do not do "$ bitbake some-package package-index" as BitBake does not schedule
the package index for after the completion of the package you are building.
Consequently, you cannot be sure of the package index including
information for the package you just built. Thus, be sure to run the
package update step separately after building any packages.
"""

So far it seems we were relying on the "create filesystem" task
triggered for image creation which would also trigger the package index
recreation, according to
https://yocto.yoctoproject.narkive.com/rjvLrVpW/question-about-rebuilding-rpm-package-index-for-updated-rpms-when-bitbake-completes
"""
The feed is normally indexed (createrepo) either when you manually run the
package-index operation, or when you construct a filesystem. Until you do that,
the feed directories are transient.
"""

Related: SYS#6023
2023-06-22 16:24:09 +02:00
Oliver Smith 7aca9c9f1e Makefile: git.osmocom.org -> gerrit.osmocom.org
Related: SYS#6022
2022-09-21 12:26:40 +02:00
Harald Welte 7c0b7a9997 Move to sysmocom poky repository
we needed to patch bitbake to make the git fetcher use https by default
in order to still be able to fetch from github repositories.  This means
we cannot continue to use upstreak poky but must use a private fork.
2022-01-11 12:35:53 +01:00
Harald Welte f02bca3f9f Makefile: Switch from git:// got https://
The unauthenticated git protocol on port 9418 is no longer supported.
https://github.blog/2021-09-01-improving-git-protocol-security-github/
2022-01-11 11:02:01 +01:00
Harald Welte e70782692b migrate from git.sysmocom.de to gitea.sysmocom.de 2021-05-16 13:18:34 +02:00
Pau Espin Pedrol bca318fdac Makefile: Add sysmocom-voice-sip-image default target for sysmobts 2021-02-02 18:13:53 +01:00
Pau Espin abf8de0bcf Revert "HACK: disable dahdi on alix/apu2 builds, too."
This reverts commit e158378a69.

Proper fix is available in meta-telephony
I6b4802374194d0fb722d0c4fd8d6ed72cd468262.

Related: SYS#4760
2019-12-23 11:40:52 +01:00
Harald Welte e158378a69 HACK: disable dahdi on alix/apu2 builds, too.
We are seeing very strange build errors, and I have not the
slightest clue how to debug those:

ERROR: dahdi-linux-2.10.2-r0 do_package_write_ipk: Error executing a python function in exec_python_func() autogenerated:

The stack trace of python calls that resulted in this exception/failure was:
File: 'exec_python_func() autogenerated', lineno: 2, function: <module>
     0001:
 *** 0002:do_package_ipk(d)
     0003:
File: '/home/oe/system-images/git/poky/meta/classes/package_ipk.bbclass', lineno: 231, function: do_package_ipk
     0227:            conffiles.close()
     0228:
     0229:        os.chdir(basedir)
     0230:        subprocess.check_output("PATH=\"%s\" %s %s %s" % (localdata.getVar("PATH"),
 *** 0231:                                                          d.getVar("OPKGBUILDCMD"), pkg, pkgoutdir), shell=True)
     0232:
     0233:        if d.getVar('IPK_SIGN_PACKAGES') == '1':
     0234:            ipkver = "%s-%s" % (d.getVar('PKGV'), d.getVar('PKGR'))
     0235:            ipk_to_sign = "%s/%s_%s_%s.ipk" % (pkgoutdir, pkgname, ipkver, d.getVar('PACKAGE_ARCH'))
File: '/usr/lib/python3.4/subprocess.py', lineno: 620, function: check_output
     0616:            process.wait()
     0617:            raise
     0618:        retcode = process.poll()
     0619:        if retcode:
 *** 0620:            raise CalledProcessError(retcode, process.args, output=output)
     0621:    return output
     0622:
     0623:
     0624:def list2cmdline(seq):
Exception: subprocess.CalledProcessError: Command 'PATH="/home/oe/system-images/build.apu2/tmp/sysroots-uninative/x86_64-linux/usr/bin:/home/oe/system-images/git/poky/scripts:/home/oe/system-images/build.apu2/tmp/work/all-poky-linux/dahdi-linux/2.10.2-r0/recipe-sysroot-native/usr/bin/i586-poky-linux:/home/oe/system-images/build.apu2/tmp/work/all-poky-linux/dahdi-linux/2.10.2-r0/recipe-sysroot/usr/bin/crossscripts:/home/oe/system-images/build.apu2/tmp/work/all-poky-linux/dahdi-linux/2.10.2-r0/recipe-sysroot-native/usr/sbin:/home/oe/system-images/build.apu2/tmp/work/all-poky-linux/dahdi-linux/2.10.2-r0/recipe-sysroot-native/usr/bin:/home/oe/system-images/build.apu2/tmp/work/all-poky-linux/dahdi-linux/2.10.2-r0/recipe-sysroot-native/sbin:/home/oe/system-images/build.apu2/tmp/work/all-poky-linux/dahdi-linux/2.10.2-r0/recipe-sysroot-native/bin:/home/oe/system-images/git/poky/bitbake/bin:/home/oe/system-images/build.apu2/tmp/hosttools" opkg-build dahdi-firmware /home/oe/system-images/build.apu2/tmp/work/qemux86-poky-linux/dahdi-linux/2.10.2-r0/deploy-ipks/all' returned non-zero exit status 127

ERROR: dahdi-linux-2.10.2-r0 do_package_write_ipk: Function failed: do_package_ipk
ERROR: Logfile of failure stored in: /home/oe/system-images/build.apu2/tmp/work/qemux86-poky-linux/dahdi-linux/2.10.2-r0/temp/log.do_package_write_ipk.787195
ERROR: Task (/home/oe/system-images/git/meta-telephony/recipes-isdn/dahdi-linux/dahdi-linux_2.10.2.bb:do_package_write_ipk) failed with exit code '1'
2019-12-12 11:53:40 +01:00
Pau Espin 6c68a99f45 cfg: Disable libosmo-abis DAHDI support for ARM BTS 2019-12-03 12:33:30 +01:00
Daniel Willmann 861c37e332 Revert "Makefile: Remove oc2g from MACHINES to prevent automatic build"
This reverts commit a3e4df7d16.
make build-all will now build the oc2g images as well
2019-02-04 12:15:23 +01:00
Daniel Willmann a3e4df7d16 Makefile: Remove oc2g from MACHINES to prevent automatic build 2019-01-29 15:55:06 +01:00
Daniel Willmann 618f660026 OC2G: Build nitb image as well and fix feed URL 2019-01-29 15:12:45 +01:00
Omar Ramadan 0ef8b7a9f6 Add OC-2G target 2019-01-29 15:12:15 +01:00
Pau Espin a0e27de3bd Fix only first BUILD_TARGET element being taken (now correct fix)
Commit d4260e1a4a fixed the issu but only
accidentally. It seems when in a Makefile you do A="xyz abc", the full token
"zyx abc" (quotes included) is subsituted by $(A). As a result, Previous
commit attempting the fix was ending up in ""xyz abc"" being used.

Let's instead remove quotes altogether.
2018-12-18 21:13:48 +01:00
Pau Espin 20eea59978 Revert "upload-build.sh: Create upload directory before using rsync"
This reverts commit d6d0c3c644.

Revert as it's not useful because the host doesn't accept ssh
connections. In this case, ssh will return exit code 0 but no work will
be done. It can be easily tested with "ssh sysmocom-downloads ls /"
which outputs nothing. Trying to acces a console through ssh errors
directly with a message.
2018-01-19 17:09:33 +01:00
Pau Espin d6d0c3c644 upload-build.sh: Create upload directory before using rsync
Otherwise first time a machine+release combination is uploaded rsync
will fail:
sync --delete -avz tmp/deploy/ipk/ sysmocom-downloads:sysmocom-apu2/201705-nightly/ipk
sending incremental file list
rsync: mkdir "/home/generic/web-files/sysmocom-apu2/201705-nightly/ipk" failed: No such file or directory (2)
2018-01-18 15:13:53 +01:00
Pau Espin e72638bdc7 Makefile: Use CURDIR instead of PWD
Fixes setup of BASE_DIR git repositories when used like: make -C
system-images update

Related: SYS#3967
2017-10-25 19:34:20 +02:00
Pau Espin 8030258191 sysmocom-apu2: Add iu to DISTRO_FEATURES
This is so far the only machine for which we need 3g support and need to
install osmo-hnbgw.
2017-10-18 11:38:22 +02:00
Pau Espin eb2b25a433 cfg: Update PACKAGE_FEED_URIS to point to feeds.sysmocom.de 2017-10-10 16:14:11 +02:00
Pau Espin bac9e0e393 Makefile: sysmobts: Build sysmocom-nitb-rauc-image by default 2017-10-06 19:46:41 +02:00
Pau Espin ea7ed240d2 Makefile: Enable build of sysmocom-nitb-image by default 2017-10-06 19:44:17 +02:00
Pau Espin e3cbf53138 Makefile: Add GIT_BRANCH_* vars 2017-10-06 19:43:42 +02:00
Pau Espin bf116e5a61 Makefile: Build core-rauc image instead of nitb-rauc
We don't plan to support NITB or old openbsc repo in 201705 for now.
Build the core images instead of the NITB, which since recently don't
contain osmo-nitb or other binaries from the openbsc repo, only osmo-pcu
and osmo-bts are included.
2017-09-08 14:37:52 +02:00
Holger Hans Peter Freyther 10236ec492 make-stable: Implement a working dry run with nicer output
We can not use --link-dest when doing the dryrun as it will consider
everything up to date. So create some base args and then add the right
parts to it. Skip --checksum from the cli as it makes things really
slow. Use rsync (instead of diff -r) as this will be used for the final
handling. Keep on using sub dirs as this gives higher control. We could
otherwise just syncrhonize everything.
2017-08-07 12:39:06 +08:00
Holger Hans Peter Freyther 30fac6acb8 make-stable: Gracefully skip directories that do not exist 2017-08-07 12:34:37 +08:00
Holger Hans Peter Freyther d413b7be38 make-stable: Extend it to more directories uploaded that need sync 2017-08-07 12:33:01 +08:00
Holger Hans Peter Freyther ccf6ece826 make-stable: Make it a proper list and add quotes and drop the ; 2017-08-07 12:32:37 +08:00
Holger Hans Peter Freyther 06d37a5847 Merge remote-tracking branch 'origin/master' into 201705 2017-08-06 23:31:49 +08:00
Holger Hans Peter Freyther a5ac1374cd upload: Not all builds have a tool directory, ignore failure
This is an intermediate band-aid for the 201705 branch and with some
more bash rules we can make this a function.
2017-08-06 23:27:51 +08:00
Holger Hans Peter Freyther b8c9e10c12 Makefile: Fix the clean target by re-ordering
The sstate cleaning requires a tmp directory. So don't delete it first.
2017-08-06 23:27:51 +08:00
Pau Espin 6c0a4e0af0 gitignore: Add downloads dir 2017-08-06 23:27:51 +08:00
Holger Hans Peter Freyther 3ddf399a96 Merge remote-tracking branch 'origin/master' into 201705 2017-08-04 18:40:08 +08:00
Holger Hans Peter Freyther 36df2b58cd Merge remote-tracking branch 'origin/master' into 201705 2017-08-04 17:09:11 +08:00
9 changed files with 65 additions and 28 deletions

View File

@ -24,19 +24,26 @@ endif
# Variables
SYSMOCOM_RELEASE=201705
POKY_RELEASE=pyro
REPOS=poky meta-telephony meta-sysmocom-bsp meta-qt5 meta-sysmocom-bsp meta-smalltalk
MACHINES=sysmobts sysmobts2100 sysmocom-apu2 sysmocom-alix
REPOS=poky meta-telephony meta-qt5 meta-sysmocom-bsp meta-smalltalk
MACHINES=sysmobts sysmobts2100 sysmocom-apu2 sysmocom-alix oc2g
FEED_NAME=$(SYSMOCOM_RELEASE)-testing
# The default targets to pick depending on machine
BUILD_TARGET_sysmobts = "meta-toolchain-osmo task-sysmocom-feed sysmocom-core-image sysmocom-nitb-rauc-image image-rauc-rescue-initramfs image-rauc-slot-initramfs image-rauc-ubi"
BUILD_TARGET_sysmobts2100 = "meta-toolchain-osmo task-sysmocom-feed sysmocom-core-image"
BUILD_TARGET_sysmocom-apu2 = "core-image-minimal-initramfs meta-toolchain-osmo task-sysmocom-feed sysmocom-core-image core-image-minimal-initramfs"
BUILD_TARGET_sysmocom-alix = "core-image-minimal-initramfs meta-toolchain-osmo task-sysmocom-feed sysmocom-core-image core-image-minimal-initramfs"
BUILD_TARGET_sysmobts = meta-toolchain-osmo task-sysmocom-feed sysmocom-core-image sysmocom-nitb-image sysmocom-voice-sip-image sysmocom-core-rauc-image sysmocom-nitb-rauc-image image-rauc-rescue-initramfs image-rauc-slot-initramfs image-rauc-ubi
BUILD_TARGET_sysmobts2100 = meta-toolchain-osmo task-sysmocom-feed sysmocom-core-image sysmocom-nitb-image
BUILD_TARGET_oc2g = meta-toolchain-osmo task-sysmocom-feed sysmocom-core-image sysmocom-nitb-image
BUILD_TARGET_sysmocom-apu2 = core-image-minimal-initramfs meta-toolchain-osmo task-sysmocom-feed sysmocom-core-image sysmocom-nitb-image core-image-minimal-initramfs
BUILD_TARGET_sysmocom-alix = core-image-minimal-initramfs meta-toolchain-osmo task-sysmocom-feed sysmocom-core-image sysmocom-nitb-image core-image-minimal-initramfs
# Pick the one depending on $@. Not sure if .SECONDEXPANSION is more
# approiate here or not.
BUILD_TARGETS="$(BUILD_TARGET_$(CUR_MACHINE))"
BUILD_TARGETS=$(BUILD_TARGET_$(CUR_MACHINE))
GIT_BRANCH_POKY = "$(POKY_RELEASE)"
GIT_BRANCH_SYSMOCOM_BSP = "$(SYSMOCOM_RELEASE)"
GIT_BRANCH_TELEPHONY = "$(SYSMOCOM_RELEASE)"
GIT_BRANCH_SMALLTALK = "$(SYSMOCOM_RELEASE)"
GIT_BRANCH_QT5 = "$(SYSMOCOM_RELEASE)"
#
usage:
@ -66,8 +73,9 @@ help:
@echo " make-server-structure - Create directories for machine/release"
@echo "Available variables:"
@echo " V=1 - Enable verbose command output"
@echo " SYSMOCOM_RELEASE=name - Pick branch during clone"
@echo " POKY_RELEASE=name - Pick branch during clone"
@echo " SYSMOCOM_RELEASE=name - Pick release branch during clone"
@echo " POKY_RELEASE=name - Pick release branch during clone"
@echo " GIT_BRANCH_* - Pick branch for a specific repository"
# Fetch/update all repos... Expand REPOS and append -update to the rule
# e.g. poky-update meta-telephony-update
@ -85,15 +93,15 @@ git:
## Clone repositories. The other option is by variable something like BRNACH_poky, REPO_poky
git/poky: | git
$(V)cd git && git clone --branch=$(POKY_RELEASE) --depth=1 git://git.yoctoproject.org/poky
$(V)cd git && git clone --branch=$(GIT_BRANCH_POKY) --depth=1 https://gitea.sysmocom.de/sysmo-bts/generic-poky poky
git/meta-sysmocom-bsp: | git
cd git && git clone --branch=$(SYSMOCOM_RELEASE) git://git.sysmocom.de/poky/meta-sysmocom-bsp
cd git && git clone --branch=$(GIT_BRANCH_SYSMOCOM_BSP) https://gitea.sysmocom.de/sysmo-bts/meta-sysmocom-bsp
git/meta-telephony: | git
cd git && git clone --branch=$(SYSMOCOM_RELEASE) git://git.osmocom.org/meta-telephony
cd git && git clone --branch=$(GIT_BRANCH_TELEPHONY) https://gerrit.osmocom.org/meta-telephony
git/meta-smalltalk: | git
cd git && git clone --branch=$(SYSMOCOM_RELEASE) git://github.com/sysmocom/meta-smalltalk
cd git && git clone --branch=$(GIT_BRANCH_SMALLTALK) https://github.com/sysmocom/meta-smalltalk
git/meta-qt5: | git
cd git && git clone --branch=$(SYSMOCOM_RELEASE) git://github.com/sysmocom/meta-qt5
cd git && git clone --branch=$(GIT_BRANCH_QT5) https://github.com/sysmocom/meta-qt5
## Create a build directory, e.g. build.sysmobts
## Use Poky to set-up the directory and then customize it. Copy files
@ -105,10 +113,10 @@ build.%: | git/poky
# Append entries to conf/local.conf. Common first, machine second... filter
$(Q)$(foreach file,$(CFG_FILES), \
cat $(file) | sed s,BASE_DIR,$(PWD), >> $@/conf/local.conf;)
cat $(file) | sed s,BASE_DIR,${CURDIR}, >> $@/conf/local.conf;)
@echo "require conf/distro/include/sysmocom-defaults.conf" >> $@/conf/local.conf
$(Q)cat cfg/bblayers.conf | sed s,BASE_DIR,$(PWD), > $@/conf/bblayers.conf
$(Q)cat cfg/bblayers.conf | sed s,BASE_DIR,${CURDIR}, > $@/conf/bblayers.conf
# generic git pull --rebase rule. Let's assume this is matching poky-update
@ -123,7 +131,7 @@ build.%: | git/poky
# Start a build..
%-build: | build.$(subst -build,,%) git/poky
$(Q)/bin/bash -c "source git/poky/oe-init-build-env build.$(CUR_MACHINE) && bitbake $(BUILD_TARGETS)"
$(Q)/bin/bash -c "source git/poky/oe-init-build-env build.$(CUR_MACHINE) && bitbake $(BUILD_TARGETS) && bitbake package-index"
%-upload: | build.$(subst -upload,,%) git/poky
$(Q)cd build.$(CUR_MACHINE) && ../scripts/upload-build.sh $(CUR_MACHINE) $(FEED_NAME)

10
cfg/oc2g/01_oc2g Normal file
View File

@ -0,0 +1,10 @@
MACHINE = "oc2g"
# feed
PACKAGE_FEED_URIS = "https://autoupdate:eechiesuboot@downloads.sysmocom.de/generic/oc2g/201705/ipk/"
# HW doesn't have rtc
MACHINE_FEATURES_BACKFILL_CONSIDERED = "rtc"
# Disable DAHDI support in libosmo-abis:
PACKAGECONFIG_remove_pn-libosmo-abis = "dahdi"

View File

@ -1,7 +1,7 @@
MACHINE = "sysmobts-v2"
# feed
PACKAGE_FEED_URIS="https://autoupdate:eechiesuboot@downloads.sysmocom.de/generic/sysmobts/201705/ipk/"
PACKAGE_FEED_URIS="https://autoupdate:eechiesuboot@feeds.sysmocom.de/generic/sysmobts/201705/ipk/"
# HW doesn't have rtc
MACHINE_FEATURES_BACKFILL_CONSIDERED = "rtc"
@ -9,3 +9,6 @@ MACHINE_FEATURES_BACKFILL_CONSIDERED = "rtc"
# Ignore old kernels
BBMASK="recipes-bsp/linux/linux_2.6.39.bb recipes-bsp/linux/linux-sysmocom_3.10.bb"
PREFERRED_VERSION_linux-sysmocom = "4.9.14+git%"
# Disable DAHDI support in libosmo-abis:
PACKAGECONFIG_remove_pn-libosmo-abis = "dahdi"

View File

@ -1,8 +1,10 @@
MACHINE = "sysmobts2100"
# feed
PACKAGE_FEED_URIS = "https://autoupdate:eechiesuboot@downloads.sysmocom.de/generic/sysmobts2100/201705/ipk/"
PACKAGE_FEED_URIS = "https://autoupdate:eechiesuboot@feeds.sysmocom.de/generic/sysmobts2100/201705/ipk/"
# HW doesn't have rtc
MACHINE_FEATURES_BACKFILL_CONSIDERED = "rtc"
# Disable DAHDI support in libosmo-abis:
PACKAGECONFIG_remove_pn-libosmo-abis = "dahdi"

View File

@ -1,5 +1,5 @@
MACHINE = "sysmocom-alix"
PACKAGE_FEED_URIS="https://autoupdate:eechiesuboot@downloads.sysmocom.de/generic/sysmocom-alix/201705/ipk/"
PACKAGE_FEED_URIS="https://autoupdate:eechiesuboot@feeds.sysmocom.de/generic/sysmocom-alix/201705/ipk/"
MACHINE_FEATURES_BACKFILL_CONSIDERED = "rtc"

View File

@ -1,4 +1,4 @@
MACHINE = "sysmocom-apu2"
PACKAGE_FEED_URIS="https://autoupdate:eechiesuboot@downloads.sysmocom.de/generic/sysmocom-apu2/201705/ipk/"
PACKAGE_FEED_URIS="https://autoupdate:eechiesuboot@feeds.sysmocom.de/generic/sysmocom-apu2/201705/ipk/"
MACHINE_FEATURES_BACKFILL_CONSIDERED = "rtc"
BBMASK="recipes-bsp/linux/linux_2.6.39.bb recipes-bsp/linux/linux-sysmocom_3.10.bb"

1
cfg/sysmocom-apu2/10_iu Normal file
View File

@ -0,0 +1 @@
DISTRO_FEATURES_append = " iu "

View File

@ -13,16 +13,29 @@ RELEASE=$2
DRYRUN=$3
if [ "x$DRYRUN" != "x" ]; then
ARGS=--dry-run
BASE_ARGS="--recursive --delete --links --verbose --dry-run "
else
BASE_ARGS="--delete -avH"
fi
DIRS=images ipk sdk tools cache-state;
DIRS="images ipk sdk tools cache-state sources cache conf"
for i in $DIRS;
do
echo "Working on $i"
rsync $ARGS --delete -avH \
--link-dest=$PWD/web-files/$MACHINE/$RELEASE-testing/$i \
if [ ! -e $PWD/web-files/$MACHINE/$RELEASE-testing/$i ]; then
echo "Skipping $i, directory doesn't exist"
echo ""
continue
fi
if [ "x$DRYRUN" != "x" ]; then
ARGS="$BASE_ARGS"
else
ARGS="$BASE_ARGS --link-dest=$PWD/web-files/$MACHINE/$RELEASE-testing/$i"
fi
echo "Checking $i"
rsync $ARGS \
web-files/$MACHINE/$RELEASE-testing/$i/ \
web-files/$MACHINE/$RELEASE/$i/
web-files/$MACHINE/$RELEASE/$i/ | egrep -v "sending incre|sent |total"
done

View File

@ -13,7 +13,7 @@ set -ex
rsync --delete -avz tmp/deploy/ipk/ sysmocom-downloads:$1/$2/ipk
rsync --delete -avz tmp/deploy/images/ sysmocom-downloads:$1/$2/images
rsync --delete -avz tmp/deploy/tools/ sysmocom-downloads:$1/$2/tools
rsync --delete -avz tmp/deploy/tools/ sysmocom-downloads:$1/$2/tools || true
rsync --delete -avz tmp/deploy/sdk/ sysmocom-downloads:$1/$2/sdk
rsync --delete -avz tmp/cache/ sysmocom-downloads:$1/$2/cache-state
rsync --delete -avz cache/ sysmocom-downloads:$1/$2/cache