dimanche 10 juillet 2016

damn you prolog, I mean damn you university

Everybody should learn prolog. It's such a wonderful piece of idea.

If you express a graph as:

transition(S, a, T)
transition(S, b, U)
transition(T, 0, V)
transition(U, 0, K)
transition(T, c, K)
transition(U, a, U)
...

mark a special node as an accepting state

accept(c, K)

then validating input is only:

validate(S, [I|IS]) :-
  accept(I,S).
validate(S, [I|IS]) :-
  transition(S,I,X),
  validate(X,IS).

how daming is that.

dimanche 22 mai 2016

Programmation par l'exemple en Caml par Louis Gacôgne - 25 euros merveilleusements depenses

Pour ceux qui ont deja des bases en programmation, et en programmation fonctionnelle, ce petit manuel est une mine d'exercises aussi concis que generaux.

arbres, reines, graphes, hasard, lambda calcul, derivation symbolique, jamais trop de details, jamais pompeux ni trop succint... genial

http://www.decitre.fr/livres/programmation-par-l-exemple-en-caml-9782729819101.html


mardi 3 mai 2016

local ssh neighbors ?

A little bash function leveraging nmap .. mostly naive and fragile.
Until I learn how to do tcp|ip discovery.

neigh_ssh ()
{ 
    sudo nmap -v -p 22 --open --host-timeout 1 192.168.1.0/25 \
  | grep --color=auto -E -o "open port 22.*[0-9]{1,3}\.[0-9]{1,3}\.[0-9]{1,3}\.[0-9]{1,3}" \
  | grep --color=auto -E -o "[0-9]{1,3}\.[0-9]{1,3}\.[0-9]{1,3}\.[0-9]{1,3}" \
  | sort \
  | uniq
}

dimanche 1 mai 2016

systemd-nspawn and noexec flag

Trying to create customized archlinux iso (archiso, releng) I hit a lot of issues, most of them 'execv? failed, permission denied'.

Later on, just trying to run a systemd-nspawn container, again, execv permission. Boot fails.

Googling didn't get me very far, except to broaden the scope from systemd-nspawn to something more general. I try to run a binary (ls) from the container subtree. Permission denied. I md5sum container/.../ls and get the same md5 as the main linux install I'm running. I thought it could be the wrong architecture or a different binary. So if two binaries refuse to run .. maybe they're treated differently from the host OS. I then realize that my container and custom archiso were on a mounted drive.

mount | grep <mountpoint> -&gt; mount .... (...,noexec, ...)

EUREKA.

udevil mounts this drive with a lot of protective flags. udevil unmount <mountpoint>; sudo mount <device> /mnt; <container>/ ... /ls works.

systemd-nspawn -b -D <container> works.

Thanks to #archlinux for nothing (beside rubber ducking ;)</container></container></device></mountpoint></mountpoint>

vendredi 22 avril 2016

mtp under [arch]linux: mtpfs vs gvfs-mtp [mount update]

Wanted to copy some file from a Moto G to my laptop. Installed mtpfs. Mounting took minutes and failed. As a last hope I tried gvfs-mtp. I don't know how to use the gvfs-* CLI though. But it's supposed to work out of the box in GTK apps. So Chromium + Ctrl-O. I can see the smartphone listed. Walk through the phone FS. It's instantaneous, snappy as I like it. But what about the CLI ?

Well gvfs-info says nothing. There's gvfs-mount, which says nothing. gvfs-mount --list outputs the mounted drives on this machine. And the last one is mtp://[usb:005,009]/ ... Hmm maybe gvfs-ls takes an URI ?

$ gvfs-ls mtp:/// # Error: The specified location is not mounted
$ gvfs-ls mtp://  # Error: The specified location is not mounted
$ gvfs-ls mtp:///sdcard # same as above

could the [usb:...] be mandatory ? odd syntax but alas

$ gvfs-ls mtp://[usb:005,009]/
Internal Storage # yay

$ gvfs-tree mtp://[usb:005,009]/ | head -n 32
mtp://[usb:005,009]/
`-- Internal Storage
    |-- .CM_Cloud
    |   |-- cm
    |   |   |-- cm_cloud_sdk_expand.db
    |   |   `-- cm_cloud_sdk_expand.db-journal
    |   `-- cmqp
    |       |-- cm_cloud_sdk_expand.db
    |       `-- cm_cloud_sdk_expand.db-journal
    |-- .DataStorage
    |   `-- ContextData.xml
    |-- .UTSystemConfig
    |   `-- Global
    |       `-- Alvin2.xml
    |-- .com.zwsoft.zwcad
    |   |-- local_Thumb
    |   |   |-- Fonts
    |   |   `-- Sample Drawings
    |   |       |-- Villa.dwg.png
    |   |       |-- ar_house_2.dwg.png
    |   |       |-- crane.dwg.png
    |   |       `-- l_1a.dwg.png
    |   |-- resources
    |   |   |-- adinit.dat
    |   |   |-- arialuni.ttf
    |   |   |-- plotstyle
    |   |   |   |-- Icad.ctb
    |   |   |   |-- Monochrome.ctb
    |   |   |   |-- zwcad-Color.stb
    |   |   |   |-- zwcad.ctb
    |   |   |   `-- zwcad.stb
    |   |   |-- zwcad.dwt

Didn't try to copy anything yet, I guess it won't be worse than doing a full http roundtrip over wifi as I did before.

ps: there was a side effect of trying gvfs through GTK file open dialog. It mounted the volume on click. Thus the ability to use gvfs-* afterwards.

So there's a few things to do in cli beforehand.

source: http://stackoverflow.com/questions/483460/how-to-mount-from-command-line-like-the-nautilus-does

$ gvfs-mount --list
Volume(0): XT1032
  Type: GProxyVolume (GProxyVolumeMonitorMTP)

This shows XT1032 (Moto G) is detected but not mounted. What we want is the following:

$ gvfs-mount --list
Volume(0): XT1032
  Type: GProxyVolume (GProxyVolumeMonitorMTP)
  Mount(0): XT1032 -&gt; mtp://[usb:005,013]/
    Type: GProxyShadowMount (GProxyVolumeMonitorMTP)
Mount(1): mtp -&gt; mtp://[usb:005,013]/
  Type: GDaemonMount

Usually to mount a 'storage' device, one uses /dev/sd*. But there's no sd* node for the XT1032.
gvfs-mount --list --info lists a bit more info (sic) (errata: --list -i)

$ Volume(0): XT1032
  Type: GProxyVolume (GProxyVolumeMonitorMTP)
  ids:
   unix-device: '/dev/bus/usb/005/013'
  activation_root=mtp://[usb:005,013]/
  themed icons:  [phone]
  symbolic themed icons:  [phone-symbolic]  [phone]
  can_mount=1
  can_eject=0
  should_automount=1

And trying with the /dev/bus/... path

$ gvfs-mount -d /dev/bus/usb/005/013
Mounted /dev/bus/usb/005/013 at /run/user/1000/gvfs/mtp:host=%5Busb%3A005%2C013%5D

$ gvfs-ls mtp://[usb:005,013]/                                                                                                                  
Internal Storage

# yay

mercredi 20 avril 2016

usb key replicate

* usb key duplicate

* partition table

  manual cgdisk

* rsync partitions

  rsync -auv --progress <source></source> <target>

* rsync caveat : hard links, sparse file

  warning: some sparse files may have huge virtual size
    rsync, will attempt to expand them fully
    i.e: docker devicemapper (60MB on disk, virtually 100G)

  warning: some programs have one fat binary with hard links
    rsync, will attempt to copy them fully
    i.e: git, which has 114 hard links with different names

* BIOS boot (ef02) partition

  dd if=/dev/sdX1 of=/dev/sdY1 bs=1M # simply

* grub

  sudo grub-install --target=i386-pc --debug --boot-directory=/<root-mountpoint>/boot/ /dev/sdY

  warning: do not mess the device names and mountpoint names as it may modify your host system grub config


* grub UUID

  grub menuentry refers to source key UUID

* adjust root UUIDs
  /etc/fstab still refers to source key UUID

  blkid /dev/sdX? &gt;&gt; /etc/fstab
  vi /etc/fstab

  (some sed-fu would be nice)</root-mountpoint></target>

mardi 5 avril 2016

nixos 16.03 live iso boot "directly" from grub

I hate extracting iso images. I prefer to run them through grub loopback. Nixos 16.03 release is a few days old, I couldn't resist. Alas, booting it this way drops me in a rescue shell.

The initrd is set up to only have one layer of virtual fs. So it assumes the squashfs nix-store would be in /mnt-root/iso and complains since it is not.

So after trying to MITM myself and mount the desired FS where they should be, I replicate the final steps of the nixos initrd init script. Kernel panic.

I'm dumbfounded, I need to be more thorough. The init scripts reads a fsinfo file (fstab-like in a different format say) to prepare the new root for the real stage 2 kernel.

btw, my mistake was to simply mount the nix-store squashfs file in /mnt-root/nix/.ro-store and then symlink /mnt-root/nix/store to it. It works only partially as the stage2 kernel find its init and proceed to run systemd but hangs after a few services doing nothing. Only option: CTRL-ALT-DEL. (I even tried to instrument stage2 systemd into booting the rescue target, hoping for a rescue shell to investigate the hang .. no luck). Then I realize that the fsinfo mounts the squashfs in nix/.ro-store BUT, also prepares a unionfs in nix/store ! Maybe stage2 is waiting for a unionfs mount in nix/store and my symlink is confusing things up.

So I insert the missing layer in there. And just sh /init. It fails horribly saying 'cannot stop udev/queue' then 'Killed'. Not thrilled.

I extract the final steps of the init script (populate /mnt-root with proc,sys,run,dev; kill a few things; restore the path for `modprobe` in the new /proc/sys/kernel/modprobe value). And then exec switch_root ...

I can now enjoy nixos 16.03 live booting directly from grub. If you allow directly to allow user intervention .. heh.

mercredi 30 mars 2016

archlinux live archiso expand cow persistent overlay

Seems that by default, archiso cow persistent space is set to 262M. Too short for some needs [1]. Linux said the cow fs was filled while installing some packages. Struggle to find a way to add some more writeable space. Then I saw in mtab fs options that cowspace being tmpfs, has a size parameter. So remounting it with different settings.

sudo mount -oremount,size=<space>k cowspace

did the job

now there's an overlay named airootfs, it will show the old size, remounting put it in sync.

sudo mount -oremount airootfs

now both will show <space>.

vendredi 26 février 2016

quick note: php v7 phar and openssl modules are now in core

If you see warnings such as below, for openssl.so or phar.so:

PHP Warning:  PHP Startup: Unable to load dynamic library '/usr/lib/php/modules/openssl.so' - /usr/lib/php/modules/openssl.so: cannot open shared object file: No such fi
le or directory in Unknown on line 0
PHP Warning:  PHP Startup: Unable to load dynamic library '/usr/lib/php/modules/phar.so' - /usr/lib/php/modules/phar.so: cannot open shared object file: No such file or
directory in Unknown on line 0

Then comment out the entries in your php.ini

source: http://unix.stackexchange.com/questions/256338/how-to-install-php7-modules-in-archlinux/256482

ps: Similar issue with mysql.so, deprecated for mysqli.so. source: https://www.digitalocean.com/company/blog/getting-ready-for-php-7/

samedi 20 février 2016

java8 nashorn javascript and clojure run into a repl

Java javascript engine (nashorn) is easily accessible through the javax.script fAPI. So I popped a clojure repl to instanciate a javascript compiler so Java could run my js bytecode.

(ns three.lisp.tower)

(def js (.getEngineByName (javax.script.ScriptEngineManager.) "nashorn"))

(defn jsc [code]
          (let [prelude "{add: function(a,b){return a+b;}, inc: function(n){return n+1;}}",
             link "(function(prelude){return (function(prelude){ %s })(prelude);})(%s)"
             emit (fn [l p s] (format l s p))
             js (fn [s] (.eval js s))]
        (js (emit link prelude code))))


user=> (jsc "return Array(1,2,3).map(prelude.inc).reduce(prelude.add)")
9.0

#EnoughProcrastination

mercredi 10 février 2016

frictionless Java with .. clojure

Late realization: you can use clojure with Java. Not with Java. For Java. We all know clojure java interop is good, but that's not what is at play(sic). See, Java is verbose (duh), compiled (duhbl) and mute. On the other hand, clojure, being a lisp with a repl, is real-time, short and generic, and doesn't even require `print` to inspect values most of the time. So instead of writing long main println-full methods, compiling, running, parsing the output ... you just import some java classes and start poking around. Frustration: gone. Even better, clojure is human friendly, has clojure.reflect/reflect to inspect classes and/or instances; clojure.pprint/pprint for quick formatting needs. If java bytecode came with docstrings and javadoc... clojure repl would be a better IDE than Eclipse. And it's probably a few lines away to scrape some remote or local html set and get going. People probably delegate that to manual browsing or a text editor extension. Anyway, my java project makefiles [1] have now a `repl:` rule to enjoy Java without Java (sorry dude).

[1] yes, I like IDE-free projects so a makefile with a few rules. I even learned about jdb, because yeah, text mode java debugging !

mardi 9 février 2016

programming theorization and design based on information content ?

instead of copying and mutating data, layer and intercept ? lazy and virtual

adding two strings yields an entity knowing about the two strings and how to be a string based on the idea of concatenation, no need to recreate a new string of both content

str s \
         ,`- str u
str t /

it's all good in systems were information chunks aren't mutated, erased by local agents

jeudi 4 février 2016

user extension quality

I used to refactor (popular) wordpress plugins. I don't do it but I use a chromium extension (not a popular one though) which I realized was using half of my ram. Pretty hot for a leak.

The source is tiny, so tiny you wonder how this thing can leak at all, there's nothing except one simple loop.

Turns out it was a piece of debugging code. The whole production source is full of them, 90% of them are harmless except that one line.

User extensions...

System performance reminders - Overheating, slow disk IO, useless network IOs

In the last few months my laptop started to act differently, annoyingly slow, annoyingly hot. I didn't sense changes, reasons for that drop in 'performances'. I was also a bit exhausted so my first reaction was: this one isn't enough, buy a new one. In the back of my mind I knew that didn't make any sense, but still.

Bit by bit issues were resolved. Here's the log:

Browser ad block

I used the usual adblock plus chrome extension. It's subpar and sucked lots of cpu cycle and memory. Recently ublock Origin appeared and requires a lot less resources. Page load 2x faster, memory usage down.

SSD hysteresis

I have an intel 320series 80GB. Configured with auto-trim (fstab under linux). Never cared about trim anymore but when I started to reach less than 1GB free space, speed dropped. Welcome back to the days of mechanical HDDs. Articles say SSD needs free space to allow for wear leveling and such. So i tried to delete the biggest most useless files. Now above 4GB free, still no normal performance. trimming is useless since it's automatic, and sudo fstrim --all --verbose reports '0 bytes trimmed'.

I thought it was a linux bug, removed auto-trim and rebooted. Still nothing. But then manually trimming yeld some output 'xxGB trimmed'. Performance is back to normal levels. Still don't know if it was a bug, where, or if it's expected behavior that  when reaching <1GB free space, you have to force trim ...?

DNS

Installed pdnsd, a DNS cache. For heavy ajax websites like twitter, page rendering was significantly faster. Using twitter made me anxious(see the last point to see why), now I'm mostly zen. (2x, 3x faster perceived speed)

Heatsink / fan

The most recent one. The machine felt hot. But when the system shutdowns for no reason you start to worry. Logs have messages with cpu issued critical warnings about reaching maximum operating temperature -> shutdown. It was often around 80degC, which I thought was alright for CPUs but anyway. Peeking under the keyboard I see no large amount of dust. After a few weeks and 3 more panic shutdowns I tear down the whole thing. Still no big blog of dust that could impede air flow. I use the house vacuum cleaner, fold bits of paper to rub through heatsink fins. Now the cpu tops at 60degC when watching 720p. and below 50 on normal use. What a relief, and a simple fix too, too easy to forget.

I'm tempted to redo it but deeper. I can renew the thermal paste between cpu and sink. And also make new 'fin cleaners' with alcohol to clean the metal surface better.
People told me even a thin layer of dust on metal can change the thermal diffusion power.

On my way I also dismantled the keyboard entirely, it was disgusting. Alcohol bathed keycaps, brushed the board. The laptop feels almost new.

The joy of 'knowing' (a little, I'm still dreaming of fixing mainboards, fans DC motor, firmware and LCD panel)

samedi 17 octobre 2015

large dynamic language spaghetti project refactoring ??

I forked a project on github. ~1000 lines of python, single file of course. I can't bear reading this kind of code, so I devised to refactor it.

I don't have a battle tested method. Of course I have some ideas, isolate functional dependencies, scopes. Gradually eliminating global variables. The issue is I did it on the fly, without a clear plan, a `framework`. Now 2 weeks in, 70% of the code base is cleanly modular now, the code run, but I saw some regressions, nothing major but still, I haven't the slightest idea of how many and how large the regressions are. That brings me back to the framework thing. Refactoring isn't an object, I understand it, it's a craft. The first idea is of course surrounding the code base with regression tests. The original project had none, no tests at all. And I only learned unit testing on OOP, not on black boxes / god module. What else can one do ? how to make the migration as logical, documented and predicatble as possible ?

smallest possible working commit

When coding I often have to fight my urges to change existing code too much. New way to write a function, decouple big into small ones, reorder them, rename variables, etc etc[1]. This is the artsy part of my brain trying to express ... but it's counter productive. I need another dogma, another fetish. Recently I landed on the idea of challenging myself to make the smallest possible set of changes to a piece of code in order to fix an issue. That includes not even fixing loose formatting. Anything that isn't directly and strictly necessary to resolve the problem shouldn't be done. yet.

Ergonomic and aesthetic modifications will be done in a following commit.

As mentioned in [2], Single responsibility principle applies here to.

[1] Or even working on many changes at once.
[2] http://programmers.stackexchange.com/questions/10793/when-is-a-version-control-commit-too-large

jeudi 8 octobre 2015

parallella headless ubuntu image default credentials

In case you're too impatient to search for them after downloading the OS image from here http://www.parallella.org/create-sdcard/

The credentials are: parallella/parallella (listed on step 5 tab https://www.parallella.org/quick-start/#tab-1416026896334-4-8)

dimanche 10 août 2014

Not so shallow changes - archlinux 64bits / c2d / ssd / wmii / fish

From firesales I got a cheap 80GB intel SSD. Put it in my ThinkPad X60s and basically saw jesus and allah doing the funky chicken. Everybody says it, but until you see it, you will underestimate the impact of a ~no latency disk drive. That old core duo suddenly boots in 5 seconds top. BIOS, Grub, two lcd blinks, and then desktop.

A few weeks later, actively procrastinating, I want to fix the laptop status LED at the bottom of the LCD. I found out it was probably the LCD cable, which doubles as LED driver, that might be partially failing. Having a x61 with a bricked motherboard on a shelf, I want to see if I can leverage its LCD cable, in the case it's compatible, which I don't know.

Reaching for this cable costs time since it requires to disassemble almost everything, but I'm going for it. I start with the x61 LCD block. Then remove the keyboard on my x60s to access the LCD cable socket on the motherboard. At that point I'm only testing compatibility on this side of the cable. I boot the x60s and the LCD works fine (it's badly shattered on 30% of the surface, but the panel is ok everywhere else, so I'm able to see the BIOS splash screen). #cool

Now I have to dismantle the panel from the lid etc etc, on both laptops. Only when everything is in pieces I realize, the cables are electrically compatible, meaning both sides are the same sockets and wires. But. But... the LCD panels are build upside down. One has the controller circuit at the bottom, the other at the top. The cables are built specifically for one kind, no way to stretch or fold it back, it's too short. #fail

Time to rollback. I rewire the old panel, ~difficultly. Plug cables and power supply, not the case screws since time taught me to screw things only after testing. No backlight on the LCD. I can see shades meaning pixels are drawn but no light. I unplugged the inverter cable a bit (with free scary static noise) while installing it in the lid. Easy to fix. 2nd boot, now the backlight works fine, so fine I can see a ~10pixel wide black band in the middle. It's regular and small so I try to poke around to see if the cable is not seated properly. No luck. I discover that below the LCD are running to very thin, very dense cables called 'tab bonds' which are the final panel data feeders. Indeed, fiddling with them fix the fringe of the black band, but it's not stable.

some links:

http://www.justanswer.com/tv-repair/73km2-2008-toshiba-lcd-v52xv545u-need-repair-when.html
http://www.badcaps.net/forum/showthread.php?t=25371
http://blog.stuffedcow.net/2012/03/thinkpad-x61-tablet-lcd-bubbles/
http://vimeo.com/3933798

The laptop works fine but that weird strip is too distracting. And that laptop was a little too slow, and 1.5GB of RAM too short. I have that other x61 c2d/4GB lying around. Let's retire the x60s, even though I love this little crippled bastard, he took everything I through at him. #soldier.

I plug the ssd in the x61, boot my old 32bit system. But I wanna try 64bit, because I don't like only having access to 3GB when there are 4. Also I noticed that software behaved a little differently in 64bit, so I format / reinstall everything.

To try things early I dont install xmonad/haskell, only wmii. It's a very small and ~limited window manager. But I realize it does what xmonad did, and the basic paradigm of stacked window titles is very easy on my mind. Much easier than the 2d split of xmonad (surely xmonad can emulate wmii but I'm not in the mood for writing haskell #noob).

So that's a first. I'm not missing xmonad in the slightest for my usage.

I also tried fish. I may not program in it, but it's light and ergonomically heaven compared to bash and even zsh.

Final note:

  • arch
  • base-devel (gcc, make...)
  • python3, guile ...
  • emacs
  • chromium
  • alsa, cmus, mpv
2.5GB (including pacman and chromium caches, so more ~2GB).

Pretty slick.

vendredi 7 mars 2014

taking notes aka compiler targetting your brain ISA

learning better by correlating one domain to another. idea into graphemes. idea into actions, into spoken words, shapes.

remember taking notes in high school. The more I paid attention to formatting the better I learned. First I pruned the content. Similar to the original meaning of lecture, I sorted and picked words differently. IIRC it was mostly based about the "newness" of a subpart of the concept discussed. The less I understand it, the more carefull I'll be when writing it down, the more space I'll allocate to this part. It's a differential overlay of my brain. The format was important too, even though I despise the term format for it's superficial conotation, it is a reflection of relationships between ideas. Most of the time it's nested (up down dimension). But there's also aside dimensions which made me use colors to denote comments or any kind of lateral relationship. In the end it looked like the handwritten projection of a DAG into a yaml/xml structure. Maybe the term ontology would fit.