[buug] vi, and working with encrypted data/files

[Michael Paoli]

references/excerpts; comments, etc. added in-line (Bcc earlier author):

> Subject: Encrypting files
> To: Michael Paoli
> Date: Sat, 2 Mar 2013 20:01:56 -0800 (PST)

> Michael.
> I looked at my O'Reilly manual for vi, and experimented with
> executing a shell command and redirecting its output to a buffer and
> a buffer though a filter to a file. It works as you said. Gnome Gedit
> as installed can't execute shell commands; though it can with a
> plugin.

Yup, e.g.:
:r !command
:w !command

> I also looked at GnuPG. (Was this the encryption/decryption
> application you use?) It seems rather complicated for what I need. I
> don't need asymmetric (public and private) keys; just a single
> passphrase to encrypt and decrypt at most several hundred lines of
> text.

Well, GnuPG does quite a bit, and does it well.  Though it may be a bit
of overkill for what you're looking to do, it also may be quite
suitable.  with GnuPG:
o You may just simply use a small subset of its capabilities
o avoid using some "one-off" encryption tool (though there are also
   others that are very well supported)
o it's very well supported, known, and understood in the
   Linux/OpenSource community, so getting support is relatively easy -
   especially if one isn't doing something too obscure.

> Password Safe sounds promising; but its Linux version is in beta-test
> only as of December. I'd rather wait awhile.
Ah well, I thought it had been ported to Linux earlier.  It's pretty
good if/when one is stuck on Microsoft Windows.  Looks like data
compatible software has also been written for other platforms - probably
just a matter of time before it's also quite well available for and
supported on Linux.

And since I first started drafting this, I also stumbled across seahorse
(was browsing/walking the (mostly) default Gnome menu on Debian 6.0.7 -
System --> Preferences --> Passwords and Encryption Keys (launches -->)
Help --> About (seahorse 2.30.1)).  That looks potentially rather to
quite promising.

> Was your trick of opening vi with /dev/null a way to avoid opening a
> file on /tmp?

No.  Using /dev/null as file argument for nvi or BSD vi, has the
advantage if I accidentally/habitually do something like:
:w
that simply writes the buffer to /dev/null, rather than an actual
ordinary file.  A nice feature of nvi or BSD vi, is that if one doesn't
give a file argument, and then does:
:w
that updates/flushes buffer to temporary file, so one can then recover
from that point.  In this case, however I want to minimize or avoid it
being written in the clear to any plain file.  File argument of
/dev/null doesn't fully cover that, but it at least partially covers
that - notably in case of use of:
:w
Note that vim behaves slightly differently, and more like classic vi,
if a file hasn't been set via argument or otherwise, it operates on "NO
FILE", and give an error if one tries to do a plain:
:w
I prefer the nvi / BSD vi behavior on that point, but one could argue
and split hairs over that.  (e.g. advantage being it gets flushed to a
file, so interim writes that one doesn't want to lose along the way,
but that are intended to be quite temporary, can be saved without need
to explicitly create some temporary file and later need to remove that
temporary file, and I suppose one could counter-argue that
:w
gives a "false" sense of security in nvi / BSD vi in such case, as it
writes it out to a temporary file, which is removed when the session is
exited, however, it does also display the filename when it's written -
which makes it pretty obvious it's a temporary file, and it does also
warn when attempting to quit in such case, unless q! or the like is
used).

nvi and BSD vi - and likewise vim, still use some temporary file(s),
notably for editor crash recovery, so one isn't totally immune in such
cases from having all or much of the file written in the clear to
ordinary file(s).

:se directory
can be used in vi/nvi/vim to give an alternative (e.g. more secure)
location for temporary files, but that may not include editor recovery
temporary files, so that may only have part of the desired effect.
Note also that
:se directory
needs to be used when vi is invoked for it to be effective - e.g. in
$HOME/.exrc or EXINIT, otherwise it's "too late" and has no effect.

> And why did you execute commands from vi by explicitly
> creating a new shell?

Well, read/wrote stuff via shell commands, rather than directly giving
any filename arguments to any form of :r, :w, or :f vi commands.  That
was so there'd be no "remembered" or "known" filename for vi, again,
so, e.g.:
:w
would still write the buffer to /dev/null and not some ordinary (or at
least not any non-temporary ordinary) file.

> You said not long ago that you do not shutdown your system for weeks
> on end. When vi exits, are the memory locations that it used for its
> buffers zeroed out? Or am I not recalling that you execute a command
> to encrypt and write to disk and explicitly clear the buffer? But of
> course the information about "undoing" must still be in memory
> somewhere.
> Just trying to understand. This sounds like it might be a good topic
> for a BUUG post. If yu have already done so, just send me the link.

vi isn't a particularly security sensitive application, so it probably
takes no extraordinary efforts to protect its data, and probably mostly
relies upon the operating system (OS) to "do the right thing", e.g.
memory shouldn't "leak" from one PID to another, usually the
application doesn't wipe its memory, but simply releases it, and the OS
clears it before reallocating it (logically similarly for files).
Besides, with virtual memory management, merely wiping memory isn't
sufficient - it needs to be locked before the data is initially
written, lest the OS may relocate it or its writes at any time, and
remnant copies may remain in memory or swap.

Remember, when working with encrypted data, notably including
decryption thereof, the key(s) to decrypt it are in memory when the
decryption is done, so there will always be at least inherent trusting
of the operating system and its memory and environment thereof (e.g. if
it's virtual, the whole OS image can generally be copied/snapshotted at
an time, and generally undetectably to the OS itself (how secure is
your cloud?)).  If the OS is only exposed to encrypted data, and never
the keys or cleartext, then the security of the encrypted data from
that OS is no less secure than the security of that encryption.

Examples - with comments:

I use ex rather than vi to make it a lot easier to otherwise similarly
show the relevant points in plan text and more efficiently, and to more
easily capture via script(1).

Without /dev/null argument, our write goes to plain file and we can
retrieve what's written via e.g. cat:
$ ex
/tmp/vi.3iohBs: new file: line 1
:0a
CLEARTEXT
.
:w
/tmp/vi.3iohBs: 1 lines, 6 characters
:!cat /tmp/vi.3iohBs
CLEARTEXT
!
:q
File is a temporary; exit will discard modifications
:q!

If we use :r with filename, vi remembers that filename and :w! will
overwrite it:
$ echo CLEARTEXT > CLEARTEXT
$ ex
/tmp/vi.y76E0o: new file: line 1
:r CLEARTEXT
CLEARTEXT: 1 lines, 5 characters
:w
CLEARTEXT exists, not written; use ! to override
:w!
CLEARTEXT: 1 lines, 6 characters
:q
$

And likewise, if the file doesn't exist when we use :w, vi simply writes
it, without warning:
$ ex
/tmp/vi.L3yMsN: new file: line 1
:r CLEARTEXT
CLEARTEXT: 1 lines, 5 characters
:!rm CLEARTEXT
File modified since last write.
!
:w
CLEARTEXT: new file: 1 lines, 6 characters
:q
$

If we read in with command (!) with :r, vi doesn't know the filename,
so :w then uses the temporary file in the case of nvi or BSD vi, or NO
FILE (error when attempting to write) for classic vi and vim.
$ ex
/tmp/vi.89OuFu: new file: line 1
:r !cat CLEARTEXT
!
CLEARTEXT
:!rm CLEARTEXT
File modified since last write.
!
:w
/tmp/vi.89OuFu: 1 lines, 6 characters
:q!
$

$ ls -iL /usr/bin/vim
311861 /usr/bin/vim
$ 2>>/dev/null find /usr/bin -follow -inum 311861 -print
/usr/bin/rvim
/usr/bin/rview
/usr/bin/vim
/usr/bin/vim.basic
/usr/bin/vimdiff
$ ln -s /usr/bin/vim ex
$

vim works relatively similarly, but doesn't have the "feature" of
defaulting to writing/flushing to a named temporary file:
$ ./ex
Entering Ex mode.  Type "visual" to go to Normal mode.
:0a
CLEARTEXT
.
:w
E32: No file name
:q
E37: No write since last change (add ! to override)
:q!
$

(egad, more vim annoyances ... "Normal" mode?  Since when is ex mode in
vi not a "normal" mode?  And why has it got to tell me "Entering Ex
mode" - if I invoke it as ex I expect it to go into ex mode - and I
don't expect it to bother to tell me it did that, just give me the ex
prompt and don't tell me you went into ex mode like I told you to do.
If it was vi mode, I wouldn't have an ex prompt now, would I?, and if
it's ex mode, I get an ex prompt, so why the waste and distraction of
the additional explicit text?  If one doesn't know how to use ex or vi,
use some other editor, don't tell me redundant useless information -
unless I quite explicitly request it.  And yes, I know about vim's
"compatible" mode - it's *not* that compatible - even in "compatible"
mode).

vim doesn't pickup the filename from :r to :w, but otherwise similar:
$ ./ex
Entering Ex mode.  Type "visual" to go to Normal mode.
:r CLEARTEXT
"CLEARTEXT" 1L, 10C
:w
E32: No file name
:w!
E32: No file name
:q
E37: No write since last change (add ! to override)
:q!
$

But with vim, and likewise with vi, once we've used filename with :w,
that becomes remembered default filename:
$ ./ex
Entering Ex mode.  Type "visual" to go to Normal mode.
:r CLEARTEXT
"CLEARTEXT" 1L, 10C
:!rm CLEARTEXT
:!rm CLEARTEXT
[No write since last change]

:w CLEARTEXT
"CLEARTEXT" [New] 1L, 10C written
:w
"CLEARTEXT" 1L, 10C written
:q

$

But using /dev/null still doesn't make "everything" safe, but merely
reduces exposure and some potential hazards:

$ ex -R /dev/null
Warning: /dev/null is not a regular file
Press Enter to continue:
/dev/null: unmodified, readonly: line 1
:0a
CLEARTEXT
.
:w
Read-only file, not written; use ! to override
:w!
/dev/null: 1 lines, 10 characters
:^Z
[1]+  Stopped                 ex -R /dev/null
$ jobs -l
[1]+  9979 Stopped                 ex -R /dev/null
$ ls -l /proc/9979/fd/*
lrwx------ 1 michael users 64 Mar  3 00:48 /proc/9979/fd/0 -> /dev/pts/4
lrwx------ 1 michael users 64 Mar  3 00:48 /proc/9979/fd/1 -> /dev/pts/4
lrwx------ 1 michael users 64 Mar  3 00:47 /proc/9979/fd/2 -> /dev/pts/4
lrwx------ 1 michael users 64 Mar  3 00:48 /proc/9979/fd/3 ->  
/var/tmp/vi.recover/vi.epZEgW
lrwx------ 1 michael users 64 Mar  3 00:48 /proc/9979/fd/4 -> /dev/null
lrwx------ 1 michael users 64 Mar  3 00:48 /proc/9979/fd/5 ->  
/var/tmp/vi.recover/recover.WU7xfW
$ ls -lL /proc/9979/fd/[35]
-rw------- 1 michael users 2048 Mar  3 00:48 /proc/9979/fd/3
-rw------- 1 michael users  434 Mar  3 00:48 /proc/9979/fd/5
$ strings /proc/9979/fd/[35] | fgrep CLEARTEXT
CLEARTEXT
$

Likewise with vim:
$ ./ex
Entering Ex mode.  Type "visual" to go to Normal mode.
:0a
CLEARTEXT
.
:w
E32: No file name
:
E501: At end-of-file
:
E501: At end-of-file
:^Z^Z^Z^Z^Z^Z^Z^Z^Z^Z
Well, bloody vim doesn't want to let me suspend it (more damn vim
annoyances), so, from another terminal:
$ ps lwwwwwp 9995
F   UID   PID  PPID PRI  NI    VSZ   RSS WCHAN  STAT TTY        TIME COMMAND
0  1003  9995  6215  20   0  29492  2328 -      S+   pts/11     0:00 ./ex
$ ls -l /proc/9995/fd/*
lrwx------ 1 michael users 64 Mar  3 00:53 /proc/9995/fd/0 -> /dev/pts/11
lrwx------ 1 michael users 64 Mar  3 00:53 /proc/9995/fd/1 -> /dev/pts/11
lrwx------ 1 michael users 64 Mar  3 00:53 /proc/9995/fd/2 -> /dev/pts/11
lrwx------ 1 michael users 64 Mar  3 00:53 /proc/9995/fd/3 ->  
/home/m/michael/tmp/VI/.swp
$ strings /home/m/michael/tmp/VI/.swp | fgrep CLEARTEXT
CLEARTEXT
$

So, using argument of /dev/null does *not* prevent the buffer data from
being written to (all) other file(s) - if nothing else, most notably
editor crash recovery file(s).

And ... don't forget about other places where the cleartext may be
buffered.  E.g., not only OS, but console, terminal, X client/server;
terminal emulation, buffer, or OS upon which such is running, etc.

Not also that ye olde classic vi has (or had), at least on export
controlled versions (and in some cases only when explicitly requested
from the vendor), an  option/command for encryption - in which case
*all* the files it wrote were encrypted (including editor crash
recovery files).  Unfortunately it used *weak* encryption.  Ye olde man
pages on the topic used to be quite vague about that - indicating the
flavor of encryption used, but not being explicit about how weak it
was.  However, since the weaknesses of that encryption are quite well
known, it's more like an "open secret" in more recent decades (e.g. for
like at least the last 20 or more years).  Now, it *would* be cool, if
open source versions of vi, namely nvi / BSD vi and (egad) vim added a
-x option and :x command again, and highly compatible with ye olde
classic vi ... *but*, with ability to configure to use relatively
arbitrary encryption (e.g.  gpg, or ye olde weak (crypt/acrypt/mcrypt)
backwards compatible with classic vi encryption, or pretty much
whatever encryption one might desire).  Note also, with ye olde classic
version of vi, it was also possible to accidentally encrypt one's file,
and not quite notice that or the key one had inadvertently typed in
until it was too late ... oops.  ;-)