File selector

File selection menus typically load a default folder depending on what the application programmer sets.

Sometimes, file selection dialogs remember what the user selected the last time they used it and automatically load that same folder. This is often helpful.

Sometimes, though, neither behavior is correct. As a programmer, guessing what the user wants your program to do is tantamount to assuming all users are stupid and you are smart, but this makes you the stupid one. You should always defer decisions to the user when you do not have information to deterministically decide what the most appropriate decision is.

Solution

The user should get to choose which behavior is most appropriate each time they use a file selector. The solution is a windowing metaphor in which the user selects between two windows. The windows are mutually exclusive. The user must choose one. The user can swap between the two windows if they change their mind.

Symbols in computer science

Computer scientists need a system of symbols to simplify and unify the expression of concepts, and thereby reduce the complexity of the development process. Programmers have been stuck using the same symbols as typists for 50 years, because, in the beginning, it was expensive to make a separate keyboard for programming and for typing, and it was impractical to re-code the software to work with a special keyboard, since the software systems that were created were hard-coded to make use of the common keyboard. As a result, people were reluctant to change keyboard layouts. As a result of that reluctance, entire generations of current programmers grew up using software and keyboards that were not designed for programming, all because our predecessors set a precedent, and this precedent has constricted our minds and prevented people from dreaming about better alternatives. Most people do not even know how the keyboard works with the operating system, so they do not question it.

Unlike the computer science discipline, the math and language disciplines, as well as many other scientific disciplines, including chemistry, created their own systems of symbols to express concepts. It is ironic, then, that the programming discipline, which uses languages extensively, has not once introduced a new symbol to simplify the concepts of programming.

Strictly speaking, this is not true. The functional programming paradigm is strictly founded on the lambda calculus. The lambda calculus, a mathematical formalism of computation, has introduced the λ (lambda) to symbolize a function in the abstract. While the mathematical field uses this symbol, the programming languages that implement the lambda calculus, such as Scheme Lisp, Common Lisp, and Clojure, do not. While the Racket programming ecosystem (based on Scheme) has made it possible to do this, it is not convenient to click on a menu and select a menu item to insert a character, and this example is but one lone exception.

λ

The keyboard has dominated the programming scene since its inception, but it was designed primarily for office workers and typists, not mathematicians or any other specialized domain of knowledge that uses its own set of symbols.

This symbolic void in computer science has encouraged language designers to re-use symbols from the standard typist set, and this simply doesn’t work. The symbols already have many meanings associated with them, and these meanings are useful as is, so overriding them to mean something else results in a slew of arbitrary decisions about which symbol is more intuitive for the concept one is trying to express, and a number of work-arounds when the programmer wants to add these symbols under their original meaning, or literally, in a string.

The lack of transferrable knowledge between programming languages is due to the lack of accessible symbols on our input devices, and a lack of formal symbols within the profession. I see this as a significant pain point for programming, and I believe this is a problem that could be addressed, but hasn’t.

Linux sucks, and there’s nothing you can do about it

Recently I started working on a startup tech business to implement some of my tech ideas. I got a work-space on campus at University of Memphis and they gave me a desk and computer to work on in a shared work-space area where other entrepreneurs are working. The computer they provided is an iMac, and it is configured to work with the campus network, and forces me to sign in using my University credentials.

In order to do my work, however, I have to have complete control over the computer to be able to install software to develop programs, so my idea was to bring an external drive with me and install a Linux distribution onto the external drive. I pulled a solid-state drive out of my desktop at home and an SSD-to-USB adapter and installed a copy of Linux Mint (which is based on Ubuntu (which is based on Debian)).

I used the iMac to install the OS, and under normal circumstances, this should be no problem. Ideally, the installer (running off of a flash drive) would run in RAM and not modify the internal drives on the iMac. Installing the OS would put the bootloader onto the external drive and put the OS installation in a separate partition after the bootloader.

Unfortunately this did not go as planned. The installer gives users the option of either installing Mint the easy way, and telling Mint to install to a specific drive, or the hard way, where the user has to know how to repartition the drive according to the needs of the OS, which requires knowing what the OS is going to do before it does it, which is a catch-22. So I picked the easy option, and I selected LVM format (Logical Volume Management) as the partition type, so that I can resize partitions easily after installing. LVM allows the user to create other partition types inside of it, which are then able to be resized easily and moved to other drives that are also formatted with LVM.

In the advanced mode, it is impossible to set the system up with LVM, as it does not allow creating partitions inside of an LVM partition.

In the easy installation mode, I selected the external drive, and Mint’s installer put the OS on the selected drive, but then it put the bootloader on the internal hard-drive on the system. It does this because the internal drives are marked as “bootable” (boot flag is set). The installer assumes that the user must want to dual-boot, so it replaces the internal bootloader. Since the iMac boots using EFI (or UEFI, which is newer), the bootloader is actually booted from the NVRAM, which is a special memory piece on the motherboard that boots devices and remembers the default device. The Mint installer decided to delete this NVRAM and replace it with its own information pointing to my external drive’s OS installation. The OS installation then would point to the internal drive’s bootloader partition. The bootloader would then let me choose which OS to boot.

Installer actions:

  • Overwrote NVME
  • Overwrote bootloader on internal drive
  • Pointed NVME to external drive
  • Pointed external drive back to internal drive’s bootloader

The dysfunctional configuration:

NVME --> external drive --> internal drive

This is *incredibly* stupid as a default. I reported this to Linux Mint on their forums, but Mint can’t fix this, because they rely on Ubuntu to provide the installer code, so this has to be reported again to Ubuntu, but they might not even fix it, even though it is an obvious logic error that has an easy fix.

Boot flag work-around

The internal drive can be set to “unbootable” during installation as a work-around for this installer bug. To do this, open up GParted and change the flags before installing. After installing, reboot into the Linux Live CD (or installed OS) and change the flags back.

Fixing the University’s Computer

I was unable to fix the University’s computer on my own. After several hours of research, the only fix to restore the NVRAM and bootloader involved logging in as an admin on the MacOS installation and running the “Startup disk” selection program and clicking on the internal drive to repair it. It requires administrator privileges. The only other option was to re-install the operating system, and this meant giving it back to the tech people, who would take 2 weeks to figure out a 2 minute problem and then probably still re-install MacOS.

Most operating systems allow users to fix bootloaders by running a tool from an installer CD or USB drive. There is no such tool for MacOS.

Luckily, I managed to get someone who knew the admin password to help me fix the computer.

After Installation

After installation, Linux Mint seemed pretty good. I have tried many other distros and had many issues after installing. Mint seemed to avoid most of them, but a few major ones showed up.

First, Mint was unable to install certain programs from the App store (Software manager) including Discord and another program, due to a missing software package that these two programs relied on. Later on this problem went away (I can’t remember why), but this was a problem out of the box.

The other major problem is the drivers. I intended to use this SSD on a few different computers, including my home desktop and my laptop, so that I can maintain a consistent development environment across different machines. Unfortunately, the drivers for hardware on several of the machines are missing or broken.

WiFi on Macbook Air 2013

The first I noticed was the WiFi driver on my laptop (Macbook Air mid-2013). Because of this, I cannot use Mint on the laptop at all. The internet is so integral to programming that this is a real problem.

Sound card on iMac

The sound card on the iMac also was (and is) not working. After doing some research on the issue, it has no known fix. The other reports of the same problem have different symptoms reported by the system programs used to diagnose the problem.

https://www.linuxquestions.org/questions/linux-hardware-18/no-sound-modules-after-installing-mint-mate-18-a-4175593442/

From the research, it becomes apparent that nobody knows what they’re doing, and fixing the problem is a mixture of political red tape, lack of responsibility, and technical incompetence.

What the real problem is

There are two real problems here:

Too many options!

First, there are too many options for how to debug the problem, and none of them are simple. Linux breaks the UNIX manifesto: “Do one thing, and do it well, and make things work together”. Debugging a problem in Linux requires users to research too much. The commands that are relevant for debugging the problems do not do one thing and do one thing well. There are too many options for how to approach solving the problem. This is a fundamental design flaw in modern UNIXy operating systems and should be a warning to future OS designers on what not to do as a developer.

Too much reading!

The other problem is that Linux’s debugging process is command-based. Tools on the command line are terribly inconsistent in their interfaces. The syntax for using the tools is unintuitive and finicky, and the presentation of information is typically organized according to the mental handicaps of the developer and is often overloaded with details that are irrelevant to the user. This requires users to memorize commands, instead of providing a way to debug configuration problems by exploring and inspecting a visual representation of the system’s configuration. While the terminal is a uniform interface, the languages and syntaxes of the programs within them are very inconsistent and require too much reading to be used efficiently.

General principle: Linux is not brain-friendly

The general principle behind “Too many options” is that Linux is not compatible with how the brain learns to do things. Likewise, the general principle behind “Too much reading” is a combination of “too much memorization” and “too many irrelevant details”. The UNIXy command lines are hard on the brain’s auditory cortex, temporal lobes, hippocampus, and prefrontal cortex (PFC), and they do not make use of the visual cortex efficiently.

What do these pieces of the brain do? From Wikipedia:

The auditory cortex is the part of the temporal lobe that processes auditory information in humans and many other vertebrates. It is a part of the auditory system, performing basic and higher functions in hearing, such as possible relations to language switching.

The temporal lobe consists of structures that are vital for declarative or long-term memory.

Auditory Cortex

The temporal lobe is involved in processing sensory input into derived meanings for the appropriate retention of visual memory, language comprehension, and emotion association.

Temporal Lobe

The hippocampus (from the Greek ἱππόκαμπος, “seahorse”) is a major component of the brain of humans and other vertebrates. Humans and other mammals have two hippocampi, one in each side of the brain. The hippocampus is part of the limbic system, and plays important roles in the consolidation of information from short-term memory to long-term memory, and in spatial memory that enables navigation.

Hippocampus

The Visual cortex is powerful

The visual cortex is the strongest part of the brain for processing data. For example, the use of videos, demonstrations, and diagrams are very effective teaching tools, and the world around us is full of visual stimuli that must be processed quickly and efficiently in order to respond quickly and orient ourselves in a complex environment. The only way to do this is with vision. A simple test: walk outside and close your eyes, and try to find your way around for a few hours. It is incredibly slow and error prone.

Vision is much more powerful than other senses:

In the brain itself, neurons devoted to visual processing number in the hundreds of millions and take up about 30 percent of the cortex, as compared with 8 percent for touch and just 3 percent for hearing.

The Vision Thing: Mainly in the Brain, Discover Magazine

The UNIX command line and most programming languages are ineffective and slow tools, because they do not make use of the brain’s ability to process visual inputs quickly in parallel and contextualize large amounts of details into a cohesive idea. The visual cortex is the largest part of the human brain, occupying almost 1/3 of the mass of the brain. Humans evolved to process visual stimuli with massive parallel processing, so it is inefficient to have to sit and read one character at a time or even one word at a time. This bottleneck is partly why GUI’s were invented.

So, the next time you pop open your command line or a text editor, ask yourself, “do I want to use 3% of my brain or 30%?”.

Linux programmers are stupid

Because of the Linux community’s inability to grasp these basic psychological concepts, Linux will forever be crap. Linux programmers are so smart that they are in fact stupid. While they sit and worry about every detail of their software’s architecture, they ignore the brain’s architecture and its limitations and abilities.

Details

  • Operating System: Linux Mint 19.2 Cinnamon
  • Cinnamon Version: 4.2.4
  • Linux Kernel: 4.15.0-66-generic

Transcript of FileSystem Conversation

Below is a transcript of the conversation I had about FileSystems on the Redox-OS Mattermost chat.

amdphreak profile image

amdphreak1:08 AM

I am researching a new option for filesystems that uses the Associative model of data (graph database model). This will solve alot of problems with maintaining packages and user files across multiple physical media, different lib versions, official and development versions (if local development files). but it will necessitate creating new userland tools and a complete redesign of shell interactivity that revolves around unique ID’s and finding things via a declarative language (similar to sql and graphQL). It will also make searching by metadata and fuzzy search terms extremely painless, and maintaining allocation of files across multiple drives will be much easier. All of this design is predicated on the usefulness of group theory/tagging/bubble clouds and I am debating what other constructs will be needed to make it intuitive. But it is clear by now that hierarchical filesystems were always a horrible idea, because data is not typically cognitively organized this way.

amdphreak profile image

amdphreak1:15 AM

For example, you would be able to locate a set of files that are candidates for what data you really want by issuing a SELECT command (without the homage to sql in the actual implementation). This will present the user with file names of potential options as well as their unique ID’s. To process the file in a command you would recognize which file you wanted from the name and then copy/paste the unique ID into the next command. It would be the search tool’s responsibility to populate a list of diff’s between the files in the event there are two files with two different names.

To clarify, it would not diff the actual file contents, only the metadata. It need only locate one property that is different between all results in order to let the user determine which file is the correct one.

The files would maintain consistency across all systems by grouping them by machine ID, and grouping by user ID.

amdphreak profile image

amdphreak1:24 AM

So you can see that the file metadata effectively makes the files unique. The user can copy files from one disk to another by specifying which disks it should exist on, just like LVM. When a user modifies a file, it gets updated on all disks (buffered, nonblocking, made authoritatively by the fastest disk and recorded to a backlog for delayed processing).

But any system call to the file API will receive only one handle to a file, even though it is on two disks.

The advantages of using Associative data model become apparent when you need to find old junk or specific copies of versions of files. You just type in what you know about the file and then either narrow down results by recognition, or by analyzing the meta-diff

Transferring files from an old user account to a new user account becomes instantly easy. You just search for files tagged “user such and such” and then copy all of them and re-merge them onto the destination machine and you never once have to touch a folder or re-integrate the files. The system does it for you because it’s building its structure from simple combinations of tags.

It astounds me how slow it is to search for files on Windows if they aren’t already indexed. And this file system idea will end the never-ending battle of deciding what the least unintuitive (read: least shitty) way to organize your files hierarchically is. Perfect hierarchies just don’t exist.(edited)

But here’s the good news. This system is backward compatible with existing programs that request files from hierarchical filesystems. The associative data model can have a file that emulates the old file system hierarchies, allowing programmers to gradually transition to the new system. Storing the traditional hierarchy in an xml file or some equivalent solution would work.

thomhuds profile image

thomhuds4:11 AM

@jackpot51 what do you think about this? Personally I have no problems with hierarchical structure and would struggle to understand anything more complex as intuitively as I understand hierarchy

fabiao profile image

fabiao4:53 AM

Are there any working implementations of this as example?

samwisefilmore profile image

samwisefilmore2:28 PM1

I think I’ve seen the blog post you’re most likely getting this from. I’d like to point out that the ideas of tagged filesystems and multi-disk filesystems are completely separable: asking for a file descriptor for a specific resource located someplace on one of the disks is not incompatible with a hierarchical tree structure. You mentioned at the beginning that such a filesystem would require fundamental shift in tooling, but I think that an implementation of tagged filesystems is possible without that using schemes on redox. I’d also like to see a working implementation of this idea. I think there’s a critical mass of developers who would be interested, somebody needs to do some work to start implementing something. Initially, I think that an implementation based on utilizing the file: provider’s filesystem as an interface over disks and stuff would be helpful for making development faster, effectively a redirection layer on top of redoxfs.1+

samwisefilmore profile image

samwisefilmore2:38 PM

You can ofc go down to a proper filesystem once a POC is created, and whatnot

zen3ger profile image

zen3ger2:51 PMPinned1Commented on samwisefilmore‘s message: I think I’ve seen the blog post you’re most likely getting this from. I’d like to point out that the ideas of tagged filesystems and multi-disk filesystems are completely separable: asking for a file descriptor for a specific resource located someplace on one of the disks is not incompatible with a hierarchical tree structure. You mentioned at the beginning that such a filesystem would require fundamental shift in tooling, but I think that an implementation of tagged filesystems is possible without that using schemes on redox. I’d also like to see a working implementation of this idea. I think there’s a critical mass of developers who would be interested, somebody needs to do some work to start implementing _something_. Initially, I think that an implementation based on utilizing the `file:` provider’s filesystem as an interface over disks and stuff would be helpful for making development faster, effectively a redirection layer on top of redoxfs.

https://www.nayuki.io/page/designing-better-file-organization-around-tags-not-hierarchies this one?1

samwisefilmore profile image

samwisefilmore2:53 PM

Yes, that one


Fri, Feb 08, 2019

zen3ger profile image

zen3ger9:59 AMPinned1

One thing regarding graph based file systems that still a question to me is their performance. If I take relational DBs and graph DBs the part where generally rDB outperforms the other is when there is a fix set of queries and the environment isn’t “too dynamic”. While it would be cool to have very flexible metadata based search engine on a file system it doesn’t necessarily means it requires a full graph FS. A separate search engine that does tag searches on top of HFS could have the advantage of both.1


Wed, Feb 13, 2019

amdphreak profile image

amdphreak5:44 PMPinned

I did not get this from a blog. I came up with this idea from scratch. However, the Brian Will videos on youtube also contain a subset of this idea (circa 2014). And there is a single lonesome research article from 2002 or 2007 (will post if I can re-find it) that has this same idea. And I just found this new video from Linux Conf 2019 (Australia) with a similiar idea: https://www.youtube.com/watch?v=wN6IwNriwHc @fabiao (edited)

amdphreak profile image

amdphreak5:55 PM

That video has more research on projects that have already implemented this idea.

Look at 6:25

They use RDBMS as their model (Relational model). I’m thinking an Associative model would be better, particularly when you want to get analytics. With this built in, it would make Redox incredibly attractive to analytics programmers who want to deploy websites.

amdphreak profile image

amdphreak6:05 PM1Commented on zen3ger‘s message: One thing regarding graph based file systems that still a question to me is their performance. If I take relational DBs and graph DBs the part where generally rDB outperforms the other is when there is a fix set of queries and the environment isn’t “too dynamic”. While it would be cool to have very flexible metadata based search engine on a file system it doesn’t necessarily means it requires a full graph FS. A separate search engine that does tag searches on top of HFS could have the advantage of both.

Thanks for the analysis. I’m not entirely familiar with Graph dbs. I know there are a few different implementations of associative data model, and Graph dbs are one kind. Egh. Need to research more.

amdphreak profile image

amdphreak6:37 PMPinned1

@thomhuds have you ever had to manage a huge library of files spanning multiple disks? Ever tried to look up a file that you know you have but don’t really remember what it was called or where you put it? This operation takes forever on a hfs, and most of the time is not possible, if you have to distinguish between thousands of similarly-named files. Have you ever tried to navigate a Windows C:\ file-system structure? Have you ever gotten fed up with Windows putting a bunch of temporary files on an external hard drive and then leaving empty directories there? This habitual lack of cleanliness is a problem with the hfs usage pattern. Have you ever tried to migrate a user’s files and settings for all of their apps from an old OS installation to a new one in any OS? Ever wonder why there is a C:\users\myuser\AppData\Local\Temp folder? Have you ever wanted to just find your files without always having to memorize where you put everything? Human memories are associative, not hierarchical. You can describe your files in heuristic terms: this is what the search function in a FS explorer attempts to do, but it does so inflexibly and poorly. When your brain thinks of a new way to differentiate between files to narrow down the search results, and sort them, you have to manually hard-code that search functionality in the current model, or restructure your entire file-system, which is impossible for developers that wish to keep a stable system. Have you noticed how the Unix file-hierarchy hasn’t changed much in forever due to developers’ fears of change, and the resulting confusion this causes for new users? I’ve noticed alot of Linuxers don’t have large collections of music or photos or videos, until they’re doing it as a business, and they don’t worry about finding those things by metadata. Metadata is what your brain remembers. Over time you will forget more and more details. Memorizing locations is inefficient and unsustainable; it is unfriendly to your brain, which is why we invented Google for the internet. People’s personal troves of data are getting much larger, as well. Because memorizing URLs is idiotic, and a hierarchical organization of the internet is unscalable. Hierarchy is unnatural for how we organize memories and therefore how we organized data, but it is natural for how we make decisions and process that data. So naturally, just to be stupid, the software industry habitually stores things in hierarchies and forces you to design software associatively (OO paradigm).

Now to top it all off, have you done those things while managing your storage space? Let’s say you want video collection on an external hard drive, and all your games on an internal one? But you want your Documents path (C:\users\Documents) to reside on the internal drive, so that all those system-generated settings and files, like game saves, and application stuff (seriously just look at a user’s Documents folder after using a few dozen apps; they clutter it up and make it entirely unusable). You need the ability to isolate space requirements from program behavior requirements.(edited)

samwisefilmore profile image

samwisefilmore8:42 PM1

I don’t think anybody is specifically arguing against the need for a relational filesystem, but having an implementation would be more likely to garner attention and serious consideration for its use in redox.

amdphreak profile image

amdphreak8:43 PM

Yeah there are implementation/s. The video above is by someone who found an old implementation and updated it a bit and tested its performance.


Thu, Feb 14, 2019

thomhuds profile image

thomhuds1:21 AMPinned1Commented on amdphreak‘s message: @thomhuds have you ever had to manage a huge library of files spanning multiple disks? Ever tried to look up a file that you know you have but don’t really remember what it was called or where you put it? This operation takes forever on a hfs, and most of the time is not possible, if you have to distinguish between thousands of similarly-named files. Have you ever tried to navigate a Windows C:\ file-system structure? Have you ever gotten fed up with Windows putting a bunch of temporary files on an external hard drive and then leaving empty directories there? This habitual lack of cleanliness is a problem with the hfs usage pattern. Have you ever tried to migrate a user’s files and settings for all of their apps from an old OS installation to a new one in any OS? Ever wonder why there is a C:\users\myuser\AppData\Local\Temp folder? Have you ever wanted to just find your files without always having to memorize where you put everything? Human memories are associative, not hierarchical. You can describe your files in heuristic terms: this is what the search function in a FS explorer attempts to do, but it does so inflexibly and poorly. When your brain thinks of a new way to differentiate between files to narrow down the search results, and sort them, you have to manually hard-code that search functionality in the current model, or restructure your entire file-system, which is impossible for developers that wish to keep a stable system. Have you noticed how the Unix file-hierarchy hasn’t changed much in forever due to developers’ fears of change, and the resulting confusion this causes for new users? I’ve noticed alot of Linuxers don’t have large collections of music or photos or videos, until they’re doing it as a business, and they don’t worry about finding those things by metadata. Metadata is what your brain remembers. Over time you will forget more and more details. Memorizing locations is inefficient and unsustainable; it is unfriendly to your brain, which is why we invented Google for the internet. People’s personal troves of data are getting much larger, as well. Because memorizing URLs is idiotic, and a hierarchical organization of the internet is unscalable. Hierarchy is unnatural for how we organize memories and therefore how we organized data, but it is natural for how we make decisions and process that data. So naturally, just to be stupid, the software industry habitually stores things in hierarchies and forces you to design software associatively (OO paradigm). Now to top it all off, have you done those things while managing your storage space? Let’s say you want video collection on an external hard drive, and all your games on an internal one? But you want your Documents path (C:\users\Documents) to reside on the internal drive, so that all those system-generated settings and files, like game saves, and application stuff (seriously just look at a user’s Documents folder after using a few dozen apps; they clutter it up and make it entirely unusable). You need the ability to isolate space requirements from program behavior requirements.

(This message is a sentence-by-sentence rebuttal; I apologise if it is hard to follow but I am on mobile and cannot easily make structural changes)

No, because I don’t put related files on different disks, because that doesn’t make any sense.

Yes, and ripgrep made it blindingly easy.

This is only really true for pictures and videos that have not been renamed; personally I go through each file and rename it based on contents before using files like these. This would take a similar amount of effort as tagging each file.

Yes, and it makes a surprising amount of sense given that it’s on Windows.

No, but if I had then that’s Windows’ fault, not the filesystem’s.

Yeah. cp ~/.config /mnt/backup/ couldn’t have been easier.

No, it says right there in the name.

I can already do this. You seem to be forgetting directories can have names; by looking at the names I can tell where everything is instantly. I never thought this would be a problem. A directory named “code” is pretty unambiguous.

With tab-completion and GUI file explorers the only fact I need to remember is that my Redox source tree is under /hd/, everything else can be figured out on the fly.

I’ve never thought of a better way to structure an existing hierarchy, so I can’t really tell if this is true.

Yes, and I have personally argued against that for a while and even tried to think of a new one.

What kind of metadata does my brain remember? To use the image example, if I had one image to select from a large group, I would definitely remember it by how it looked and not time taken etc.

Forgetting details only matters if they’re not written in the file metadata.

(Cut to the next paragraph; what little of this section that is actually about filesystems has already been covered)

Done all what things?

(this sentence only has meaning in conjunction with the next)

(this sentence is incomplete and I don’t understand what its point is)

I don’t understand this sentence – what are program behaviour requirements and how are they not isolated already?(edited)

Character Map update

Come vote for a remake of the character map app here:
https://aka.ms/AA5t0ad

The character map app on Windows is incredibly useful. For adding characters from other languages, such as the ö and ü of Germanic languages and the é and ñ of Spanish and ç of French, it is absolutely a crucial application. While you can swap keyboard layouts or get specialized keyboards for these use-cases, the character map goes above and beyond this scenario, displaying every character for a given type-script like Times New Roman.

However, the app is very old, dating back to the Win32 era, so it does not work well with high resolution displays, and its functionality has not been improved upon since Windows 95 or Windows 98 (to the best of my memory).

A nice addition to the Windows ecosystem would be a pop-up interface that shows up when you click a key combination, just like the emoji keyboard (Win+.) that lets you search the character map for character variants or similar-looking or similar-sounding characters.

I have submitted a Feedback message to Microsoft asking them to update this app. Please visit the link above to vote for my change.

Set Theory in Programming

In high school and college math courses, we learn about ways to express groups of items as subsets and supersets. This branch of mathematics is called set theory.

The expression,

X ∈ A

means “Item X exists in set A”, or “X is an element of set A”. The symbol looks like an E, which is a helpful mnemonic, but it doesn’t visually convey its meaning very well. I thought it should be redesigned, so I stepped up to the task in another blog post: “Set Membership” Symbol Redesign.

Set theory is a powerful tool, and it is becoming more important for every-day computer science. It is used to describe things that need to be processed via list-comprehensions in functional languages as well as other languages.

Haskell and functions

The following is a pair of commands in Haskell that creates a list of numbers and performs a calculation on that list to generate a new list. This is called a list-comprehension.

list_1to100 = [  1..100  ]
list_2to200 = [  a*2  |  a <- list_1to100  ]

The first statement can be read as

Define list_1to100 as a list from 1 to 100, but don’t actually generate a list from 1 to 100, yet. We just want to know it can be done.

This doesn’t physically create the list. It just lets the computer know it should be able to do this, if requested later on. It is easy to fail to grasp the implications of this, so re-read this a few times before reading on.

The computer waits until the invocation to evaluate the variable. This divorce between definition and execution is called Lazy Evaluation.

The second statement,

list_2to200 = [  a*2  |  a <- list_1to100  ]

can be thought of in terms of looping constructs that are familiar to structured (ie. C-like) languages. It would go something like this:

Loop through list_1to100 and calculate the value a*2 for each item, a, in the list and store it into list_2to200.

But this is not completely accurate. Since Haskell is lazily evaluated, it can be more accurately read as:

Define list_2to200 as the expression a*2 and define a as a temporary variable that iterates through every element of list_1to100.

You then tell the computer to calculate these values by executing the name of the variable (“invoking” it), causing a chain reaction of calculations, whereas in C-like languages, invoking a variable simply coughs up a stored value from memory, which must be populated beforehand by a function. Thus, in lazily evaluated functional languages, the concepts of functions and data are interchangeable. This is the concept of first-class functions, and we say that “functions are first-class” in functional languages.

For a deeper exploration of functions in C-like languages, read my blog post: Functions in C-like Languages.

Do. Or do not. There is no trie!

During my journey to educate myself on computer science topics that were not covered in my undergraduate University curriculum, I have come across a number of new concepts that have eluded the general discourse in computer programming, one of which is the “trie” structure, pronounced “tree” or “try” (to differentiate it orally). According to the person who defined it:

As defined by me, nearly 50 years ago, it is properly pronounced “tree” as in the word “retrieval”. At least that was my intent when I gave it the name “Trie”. The idea behind the name was to combine reference to both the structure (a tree structure) and a major purpose (data storage and retrieval).

Edward Fredkin, 31 July 2008

Nice trie, but I’m not buying it.

Because of this lackadaisical naming, nobody has a clue what a “trie” is, or how to pronounce it, just by looking at its name. This failure to name concepts properly is a pervasive problem in the computer science industry and has a major toll on our ability to study and learn the subject in conjunction with all the other topics and domain specific knowledge we must learn. A data structure that is this critical to computer science should have a more intuitive name!

So, the next time you are in a position to name something well, don’t just “trie”. Do. Or do not. There is no “trie”!

Better Names

According to Wikipedia, other names for this structure are digital tree, radix tree or prefix tree. The concept of a radix, outside the context of number theory, is opaque and ill-defined and should not be used. “Digital tree” is also nondescript and should have been immediately removed from computer science vocabulary. Every tree represented on a computer is a “digital tree”. This one is no more or less “tree-y” than the others. Of the three, Prefix tree is the most sensible name.

The prefix tree is a natural choice for dictionaries, since it allows you to validate whether a series of characters is a valid string. In general, it allows one to store and validate unique paths discovered in any kind of sequential data. This data structure is a natural choice for modeling unique decision paths taken in turn-based games, such as chess. It would make sense to call this a “dictionary tree” or “sequence tree”.

What is a prefix tree/trie?

A prefix tree is a type of tree data structure that is designed to store strings of characters by organizing the characters based on their common predecessor.

[a,n,t]
[a,u,n,t]
Prefix Tree

For example, the words “ant” and “aunt” share the same first letter, ‘a’, which is their common predecessor. As each letter in the string [a, n, t] is read, it is added as a new node. Each subsequent letter is connected to the previous letter, forming a path along the branches. When adding the second string, [a, u, n, t], the process starts over from the root node and each node is checked to see if the letter’s node already exists. Since the ‘a’ node already exists as the top level node, it is traversed, and the next character is read from the input string. If the letter is not identified, it is added as a new node, creating a new branch.

When a word is finished, the last node is marked as an endpoint (leaf). The endpoint retains its mark, even if storing a new word adds child nodes to it.

In general, this kind of organization is termed a “parent” and “child” relationship, in which the node that comes after a node is its “child”, and the original node is the “parent”.

When more strings are added, if the new string starts with a different character, it becomes necessary to keep track of all top level nodes using a root node.

Root node is implied

Ordering is not automatic in the data structure. When new paths are encountered, they are stored at the end of the data structure. For small prefix trees, it is not necessary to sort the structure. However, implementations that are built for high capacity and high speed searches on a slow medium such as a hard drive or SSD must sort nodes. This means sacrificing some adding speed, when searches are concurrently being done.

Ordering is not automatic

Wide Gamut Monitor Support on Windows 10

Wide-gamut monitors are becoming ever more popular, but Microsoft has not made Windows operating system easy to use with wide-gamut monitors and wide-gamut color spaces. sRGB is no longer a given, so this needs to be remedied.

Windows Color Management System (WCS)

The current Windows Color Management System is miserable. It is completely missing documentation and has multiple separate settings that all seem to describe the same aspect of the color-managed workflow, creating ambiguity and making it impossible to understand which setting affects what. Also, the link to “Understanding color management settings” in the Color management dialog box links to a completely off-topic web page. Further research into locating the documentation leads to the developer portal with hit and miss information and no concise answers or tutorials.

PC vs TV and Cinema industries

This half-hearted handling of color management exists in spite of a decade’s worth of advances in display technology and media profession workflows. 4K movies and Youtube have necessitated an industry shift in attention to video specifications and the experience of video. Likewise, a shift in attention to color space is under way. While the computer graphics color spaces have only recently been updated, the cinema and home media industries have had a competent range of colors beginning with the advent of HDTV broadcasting and Blu-ray, which use the Rec. 709 standard. For digital cinema recordings, the DCI-P3 color space was introduced in 2010. It has been seven years since this advancement. Now, it is time to bring this positive momentum over from the media and cinema industries to the photography and creative industries. Monitors and their operating systems are in dire need to advance to a modern standard. The color space of computer monitors has not changed since the birth of sRGB in 1996 and its standardization in 1999. It is no stretch of the imagination to see that display technology and professional and consumer expectations of graphical qualities have changed since 21 years ago.

PC Displays

This technological shift is being pioneered by Apple and none other than Microsoft, themselves. Many Apple products have transitioned to using Display-P3 capable displays. Display-P3 is a form of the DCI-P3 color space adapted for use on LCD and OLED displays, or any display that emits its own light, rather than reflecting it (specifically, the gamma was adapted from the DCI-P3 standard of 2.4 to the sRGB standard of 2.2). In October 2016, Microsoft’s Surface Studio desktop computer was unveiled, featuring P3 wide color.

This raises a question: if Microsoft isn’t able to document their own color management system, and nobody truly understands how it functions, how are they diving head-first into releasing devices with wide-gamut displays? Are these devices simply using distorted colors? If not, then, conversely, if they know how the color management system works, then where is the documentation?

PROPOSED SOLUTION

Improve documentation.

Revamp the Windows Color Management GUI (and service if necessary) to intuitively reflect the color-managed workflow. The current setting names are ambiguous and do not describe the role of the setting accurately. Also, integrate the “System Defaults” settings into the main settings. It is senselessly buried in the back of the settings as a pop-up, but is just as necessary. I realize there is a separation for the purpose of locking down the system defaults from changes by unprivileged users; this may need some consideration

“Set Membership” Symbol Redesign

Set Membership Symbol

Left/Right Comparison

Set Theory

In high school and college math courses, we learn about ways to express groups of items as subsets and supersets. This branch of mathematics is called set theory.

The expression,

X ∈ A

means “Item X exists in set A”, or “X is an element of set A”. The symbol looks like an E, which is a helpful mnemonic, but it doesn’t visually convey its meaning very well. I thought it should be redesigned, so I stepped up to the task, and it appears my answer may be good enough, and not conflict with any existing symbol. The dot in my symbol represents an item being enveloped by a set or classification, while the C represents an enveloping group. The open end on the right gives the symbol a sense of imbalance, implying a sense of directionality between the terms on its left and right. The terms cannot be interchanged, due to the symbol lacking the property of commutativity.

Set theory is a powerful tool, and it is becoming more important for every-day computer science. It is used to describe things that need to be processed via list-comprehensions in functional languages as well as other languages.

I have written at length on the topic in my blog post: Set Theory in Programming.

Downloads

https://drive.google.com/open?id=108M1uB3h8YQm0AC1t321pHffi-ZvFvb_

Other symbols and Alt-codes

https://altcodeunicode.com/alt-codes-math-symbols-set-membership-empty-sets/