From Michael.Paoli at cal.berkeley.edu Thu Mar 7 13:42:56 2013 From: Michael.Paoli at cal.berkeley.edu (Michael Paoli) Date: Thu, 07 Mar 2013 13:42:56 -0800 Subject: [buug] vi, and working with encrypted data/files In-Reply-To: <1362283316.98866.YahooMailClassic@web181404.mail.ne1.yahoo.com> References: <1362283316.98866.YahooMailClassic@web181404.mail.ne1.yahoo.com> Message-ID: <20130307134256.20533z3g858udbkg@webmail.rawbw.com> references/excerpts; comments, etc. added in-line (Bcc earlier author): > Subject: Encrypting files > To: Michael Paoli > Date: Sat, 2 Mar 2013 20:01:56 -0800 (PST) > Michael. > I looked at my O'Reilly manual for vi, and experimented with > executing a shell command and redirecting its output to a buffer and > a buffer though a filter to a file. It works as you said. Gnome Gedit > as installed can't execute shell commands; though it can with a > plugin. Yup, e.g.: :r !command :w !command > I also looked at GnuPG. (Was this the encryption/decryption > application you use?) It seems rather complicated for what I need. I > don't need asymmetric (public and private) keys; just a single > passphrase to encrypt and decrypt at most several hundred lines of > text. Well, GnuPG does quite a bit, and does it well. Though it may be a bit of overkill for what you're looking to do, it also may be quite suitable. with GnuPG: o You may just simply use a small subset of its capabilities o avoid using some "one-off" encryption tool (though there are also others that are very well supported) o it's very well supported, known, and understood in the Linux/OpenSource community, so getting support is relatively easy - especially if one isn't doing something too obscure. > Password Safe sounds promising; but its Linux version is in beta-test > only as of December. I'd rather wait awhile. Ah well, I thought it had been ported to Linux earlier. It's pretty good if/when one is stuck on Microsoft Windows. Looks like data compatible software has also been written for other platforms - probably just a matter of time before it's also quite well available for and supported on Linux. And since I first started drafting this, I also stumbled across seahorse (was browsing/walking the (mostly) default Gnome menu on Debian 6.0.7 - System --> Preferences --> Passwords and Encryption Keys (launches -->) Help --> About (seahorse 2.30.1)). That looks potentially rather to quite promising. > Was your trick of opening vi with /dev/null a way to avoid opening a > file on /tmp? No. Using /dev/null as file argument for nvi or BSD vi, has the advantage if I accidentally/habitually do something like: :w that simply writes the buffer to /dev/null, rather than an actual ordinary file. A nice feature of nvi or BSD vi, is that if one doesn't give a file argument, and then does: :w that updates/flushes buffer to temporary file, so one can then recover from that point. In this case, however I want to minimize or avoid it being written in the clear to any plain file. File argument of /dev/null doesn't fully cover that, but it at least partially covers that - notably in case of use of: :w Note that vim behaves slightly differently, and more like classic vi, if a file hasn't been set via argument or otherwise, it operates on "NO FILE", and give an error if one tries to do a plain: :w I prefer the nvi / BSD vi behavior on that point, but one could argue and split hairs over that. (e.g. advantage being it gets flushed to a file, so interim writes that one doesn't want to lose along the way, but that are intended to be quite temporary, can be saved without need to explicitly create some temporary file and later need to remove that temporary file, and I suppose one could counter-argue that :w gives a "false" sense of security in nvi / BSD vi in such case, as it writes it out to a temporary file, which is removed when the session is exited, however, it does also display the filename when it's written - which makes it pretty obvious it's a temporary file, and it does also warn when attempting to quit in such case, unless q! or the like is used). nvi and BSD vi - and likewise vim, still use some temporary file(s), notably for editor crash recovery, so one isn't totally immune in such cases from having all or much of the file written in the clear to ordinary file(s). :se directory can be used in vi/nvi/vim to give an alternative (e.g. more secure) location for temporary files, but that may not include editor recovery temporary files, so that may only have part of the desired effect. Note also that :se directory needs to be used when vi is invoked for it to be effective - e.g. in $HOME/.exrc or EXINIT, otherwise it's "too late" and has no effect. > And why did you execute commands from vi by explicitly > creating a new shell? Well, read/wrote stuff via shell commands, rather than directly giving any filename arguments to any form of :r, :w, or :f vi commands. That was so there'd be no "remembered" or "known" filename for vi, again, so, e.g.: :w would still write the buffer to /dev/null and not some ordinary (or at least not any non-temporary ordinary) file. > You said not long ago that you do not shutdown your system for weeks > on end. When vi exits, are the memory locations that it used for its > buffers zeroed out? Or am I not recalling that you execute a command > to encrypt and write to disk and explicitly clear the buffer? But of > course the information about "undoing" must still be in memory > somewhere. > Just trying to understand. This sounds like it might be a good topic > for a BUUG post. If yu have already done so, just send me the link. vi isn't a particularly security sensitive application, so it probably takes no extraordinary efforts to protect its data, and probably mostly relies upon the operating system (OS) to "do the right thing", e.g. memory shouldn't "leak" from one PID to another, usually the application doesn't wipe its memory, but simply releases it, and the OS clears it before reallocating it (logically similarly for files). Besides, with virtual memory management, merely wiping memory isn't sufficient - it needs to be locked before the data is initially written, lest the OS may relocate it or its writes at any time, and remnant copies may remain in memory or swap. Remember, when working with encrypted data, notably including decryption thereof, the key(s) to decrypt it are in memory when the decryption is done, so there will always be at least inherent trusting of the operating system and its memory and environment thereof (e.g. if it's virtual, the whole OS image can generally be copied/snapshotted at an time, and generally undetectably to the OS itself (how secure is your cloud?)). If the OS is only exposed to encrypted data, and never the keys or cleartext, then the security of the encrypted data from that OS is no less secure than the security of that encryption. Examples - with comments: I use ex rather than vi to make it a lot easier to otherwise similarly show the relevant points in plan text and more efficiently, and to more easily capture via script(1). Without /dev/null argument, our write goes to plain file and we can retrieve what's written via e.g. cat: $ ex /tmp/vi.3iohBs: new file: line 1 :0a CLEARTEXT . :w /tmp/vi.3iohBs: 1 lines, 6 characters :!cat /tmp/vi.3iohBs CLEARTEXT ! :q File is a temporary; exit will discard modifications :q! If we use :r with filename, vi remembers that filename and :w! will overwrite it: $ echo CLEARTEXT > CLEARTEXT $ ex /tmp/vi.y76E0o: new file: line 1 :r CLEARTEXT CLEARTEXT: 1 lines, 5 characters :w CLEARTEXT exists, not written; use ! to override :w! CLEARTEXT: 1 lines, 6 characters :q $ And likewise, if the file doesn't exist when we use :w, vi simply writes it, without warning: $ ex /tmp/vi.L3yMsN: new file: line 1 :r CLEARTEXT CLEARTEXT: 1 lines, 5 characters :!rm CLEARTEXT File modified since last write. ! :w CLEARTEXT: new file: 1 lines, 6 characters :q $ If we read in with command (!) with :r, vi doesn't know the filename, so :w then uses the temporary file in the case of nvi or BSD vi, or NO FILE (error when attempting to write) for classic vi and vim. $ ex /tmp/vi.89OuFu: new file: line 1 :r !cat CLEARTEXT ! CLEARTEXT :!rm CLEARTEXT File modified since last write. ! :w /tmp/vi.89OuFu: 1 lines, 6 characters :q! $ $ ls -iL /usr/bin/vim 311861 /usr/bin/vim $ 2>>/dev/null find /usr/bin -follow -inum 311861 -print /usr/bin/rvim /usr/bin/rview /usr/bin/vim /usr/bin/vim.basic /usr/bin/vimdiff $ ln -s /usr/bin/vim ex $ vim works relatively similarly, but doesn't have the "feature" of defaulting to writing/flushing to a named temporary file: $ ./ex Entering Ex mode. Type "visual" to go to Normal mode. :0a CLEARTEXT . :w E32: No file name :q E37: No write since last change (add ! to override) :q! $ (egad, more vim annoyances ... "Normal" mode? Since when is ex mode in vi not a "normal" mode? And why has it got to tell me "Entering Ex mode" - if I invoke it as ex I expect it to go into ex mode - and I don't expect it to bother to tell me it did that, just give me the ex prompt and don't tell me you went into ex mode like I told you to do. If it was vi mode, I wouldn't have an ex prompt now, would I?, and if it's ex mode, I get an ex prompt, so why the waste and distraction of the additional explicit text? If one doesn't know how to use ex or vi, use some other editor, don't tell me redundant useless information - unless I quite explicitly request it. And yes, I know about vim's "compatible" mode - it's *not* that compatible - even in "compatible" mode). vim doesn't pickup the filename from :r to :w, but otherwise similar: $ ./ex Entering Ex mode. Type "visual" to go to Normal mode. :r CLEARTEXT "CLEARTEXT" 1L, 10C :w E32: No file name :w! E32: No file name :q E37: No write since last change (add ! to override) :q! $ But with vim, and likewise with vi, once we've used filename with :w, that becomes remembered default filename: $ ./ex Entering Ex mode. Type "visual" to go to Normal mode. :r CLEARTEXT "CLEARTEXT" 1L, 10C :!rm CLEARTEXT :!rm CLEARTEXT [No write since last change] :w CLEARTEXT "CLEARTEXT" [New] 1L, 10C written :w "CLEARTEXT" 1L, 10C written :q $ But using /dev/null still doesn't make "everything" safe, but merely reduces exposure and some potential hazards: $ ex -R /dev/null Warning: /dev/null is not a regular file Press Enter to continue: /dev/null: unmodified, readonly: line 1 :0a CLEARTEXT . :w Read-only file, not written; use ! to override :w! /dev/null: 1 lines, 10 characters :^Z [1]+ Stopped ex -R /dev/null $ jobs -l [1]+ 9979 Stopped ex -R /dev/null $ ls -l /proc/9979/fd/* lrwx------ 1 michael users 64 Mar 3 00:48 /proc/9979/fd/0 -> /dev/pts/4 lrwx------ 1 michael users 64 Mar 3 00:48 /proc/9979/fd/1 -> /dev/pts/4 lrwx------ 1 michael users 64 Mar 3 00:47 /proc/9979/fd/2 -> /dev/pts/4 lrwx------ 1 michael users 64 Mar 3 00:48 /proc/9979/fd/3 -> /var/tmp/vi.recover/vi.epZEgW lrwx------ 1 michael users 64 Mar 3 00:48 /proc/9979/fd/4 -> /dev/null lrwx------ 1 michael users 64 Mar 3 00:48 /proc/9979/fd/5 -> /var/tmp/vi.recover/recover.WU7xfW $ ls -lL /proc/9979/fd/[35] -rw------- 1 michael users 2048 Mar 3 00:48 /proc/9979/fd/3 -rw------- 1 michael users 434 Mar 3 00:48 /proc/9979/fd/5 $ strings /proc/9979/fd/[35] | fgrep CLEARTEXT CLEARTEXT $ Likewise with vim: $ ./ex Entering Ex mode. Type "visual" to go to Normal mode. :0a CLEARTEXT . :w E32: No file name : E501: At end-of-file : E501: At end-of-file :^Z^Z^Z^Z^Z^Z^Z^Z^Z^Z Well, bloody vim doesn't want to let me suspend it (more damn vim annoyances), so, from another terminal: $ ps lwwwwwp 9995 F UID PID PPID PRI NI VSZ RSS WCHAN STAT TTY TIME COMMAND 0 1003 9995 6215 20 0 29492 2328 - S+ pts/11 0:00 ./ex $ ls -l /proc/9995/fd/* lrwx------ 1 michael users 64 Mar 3 00:53 /proc/9995/fd/0 -> /dev/pts/11 lrwx------ 1 michael users 64 Mar 3 00:53 /proc/9995/fd/1 -> /dev/pts/11 lrwx------ 1 michael users 64 Mar 3 00:53 /proc/9995/fd/2 -> /dev/pts/11 lrwx------ 1 michael users 64 Mar 3 00:53 /proc/9995/fd/3 -> /home/m/michael/tmp/VI/.swp $ strings /home/m/michael/tmp/VI/.swp | fgrep CLEARTEXT CLEARTEXT $ So, using argument of /dev/null does *not* prevent the buffer data from being written to (all) other file(s) - if nothing else, most notably editor crash recovery file(s). And ... don't forget about other places where the cleartext may be buffered. E.g., not only OS, but console, terminal, X client/server; terminal emulation, buffer, or OS upon which such is running, etc. Not also that ye olde classic vi has (or had), at least on export controlled versions (and in some cases only when explicitly requested from the vendor), an option/command for encryption - in which case *all* the files it wrote were encrypted (including editor crash recovery files). Unfortunately it used *weak* encryption. Ye olde man pages on the topic used to be quite vague about that - indicating the flavor of encryption used, but not being explicit about how weak it was. However, since the weaknesses of that encryption are quite well known, it's more like an "open secret" in more recent decades (e.g. for like at least the last 20 or more years). Now, it *would* be cool, if open source versions of vi, namely nvi / BSD vi and (egad) vim added a -x option and :x command again, and highly compatible with ye olde classic vi ... *but*, with ability to configure to use relatively arbitrary encryption (e.g. gpg, or ye olde weak (crypt/acrypt/mcrypt) backwards compatible with classic vi encryption, or pretty much whatever encryption one might desire). Note also, with ye olde classic version of vi, it was also possible to accidentally encrypt one's file, and not quite notice that or the key one had inadvertently typed in until it was too late ... oops. ;-) From Michael.Paoli at cal.berkeley.edu Fri Mar 15 11:59:13 2013 From: Michael.Paoli at cal.berkeley.edu (Michael Paoli) Date: Fri, 15 Mar 2013 11:59:13 -0700 Subject: [buug] BALUG Tu 2013-03-19: Stefano Maffulli on OpenStack, 2013-04-16: Wikimedia Foundation; & other BALUG News Message-ID: <20130315115913.48914g4a2jw38l2c@webmail.rawbw.com> BALUG Tu 2013-03-19: Stefano Maffulli on OpenStack, 2013-04-16: Wikimedia Foundation; & other BALUG News ------------------------------ items, details further below: BALUG Tu 2013-03-19: Stefano Maffulli on OpenStack BALUG Tu 2013-04-16: Wikimedia Foundation Saltstack BOF TODAY 5pm at PyCon (Santa Clara), etc. CDs/DVDs, and other "door prizes", etc. volunteering to help BALUG Twitter https://twitter.com/#!/BALUG_org ------------------------------ Bay Area Linux User Group (BALUG) meeting Tuesday 6:30 P.M. 2013-03-19 For our 2013-03-19 BALUG meeting, we're proud to present: Stefano Maffulli[1] on OpenStack[2] - Open source software for building private and public clouds. Stafano's talk/presentation to include at least: An overview of OpenStack The history of the project, how it started, what problems it tries to solve, who's participating in the development of OpenStack and where the OpenStack Foundation[3] wants to take it. About OpenStack: OpenStack is a global collaboration of developers and cloud computing technologists producing the ubiquitous open source cloud computing platform for public and private clouds. The project aims to deliver solutions for all types of clouds by being simple to implement, massively scalable, and feature rich. The technology consists of a series of interrelated projects[4] delivering various components for a cloud infrastructure solution. About Stefano Maffulli: As OpenStack community manager, Stefano built his career around Free Software and open source: from pre-sales engineer and product manager at Italian GNU/Linux distribution MadeInLinux[5] to Italian Chancellor of the Free Software Foundation Europe[6], where he also created the FSFE Fellowship participation program. Later as community manager of leading mobile open source sync solution Funambol[7], his efforts boosted downloads and bolstered enterprise contributions. For Twitter[8], he led efforts to expand in the Italian market. In his spare time, he builds furniture and is learning how to sail in the San Francisco Bay. If you have a specific request or question for Stefano to cover, mention it on the BALUG-Talk List[9][10], or drop me a note at and I'll pass it along. 1. http://www.openstack.org/blog/author/stefano/ 2. http://www.openstack.org/ 3. http://www.openstack.org/foundation/ 4. http://www.openstack.org/projects/ 5. http://web.archive.org/web/20031014062555/http://www.madeinlinux.com/ 6. http://fsfe.org/index.en.html 7. http://www.funambol.com/ 8. https://twitter.com/ 9. http://lists.balug.org/listinfo.cgi/balug-talk-balug.org 10. http://lists.balug.org/pipermail/balug-talk-balug.org/2013-March/005017.html Please RSVP if you're planning to come (see further below). So, if you'd like to join us please RSVP to: rsvp at balug.org **Why RSVP??** Well, don't worry we won't turn you away, but the RSVPs really help BALUG and the Four Seas Restaurant plan the meal and meeting, and with sufficient attendance, they also help ensure that we'll be able to eat upstairs in the private banquet room. Meeting Details... 6:30pm Tuesday, March 19th, 2012 2013-03-19 Four Seas Restaurant http://www.fourseasr.com/ 731 Grant Ave. San Francisco, CA 94108 Easy PARKING: Portsmouth Square Garage at 733 Kearny: http://www.sfpsg.com/ Cost: The meetings are always free, but for dinner, for your gift of $13 cash or more, we give you a gift of dinner - joining us for a yummy family-style Chinese dinner - tax and tip included (your gift also helps in our patronizing the restaurant venue and helping to defray BALUG costs such as treating our speakers to dinner). ------------------------------ For our 2013-04-16 BALUG meeting, we're proud to present: The Wikimedia Foundation[1] The Wikimedia Foundation, Inc. is a nonprofit charitable organization dedicated to encouraging the growth, development and distribution of free, multilingual content, and to providing the full content of these wiki-based projects to the public free of charge. The Wikimedia Foundation operates some of the largest collaboratively edited reference projects in the world, including Wikipedia[2], a top-ten internet property. 1. http://wikimediafoundation.org/ 2. http://www.wikipedia.org/ ------------------------------ Saltstack BOF TODAY 5pm at PyCon (Santa Clara), etc. And in follow-up from Corey Quinn's Salt talk/presentation at the 2012-05-15 BALUG meeting[2], one of our attendees is now also part of the core team of developers and also passes along notice[3] of a Saltstack BOF happening TODAY at 5pm[1] at PyCon in Santa Clara. 1. http://lanyrd.com/2013/saltpy/ 2. http://lists.balug.org/pipermail/balug-announce-balug.org/2012-May/000189.html 3. http://lists.balug.org/pipermail/balug-talk-balug.org/2013-March/005019.html ------------------------------ CDs/DVDs, and other "door prizes", etc. Goodies we'll have at the BALUG meeting (at least the following): CDs/DVDs, etc. - have a peek here: http://www.wiki.balug.org/wiki/doku.php?id=balug:cds_and_images_etc We may also be able to "burn" images per request or copy to USB flash, etc. Donations of blank or +-RW media, USB flash, or funding thereof, also appreciated. See the above URL for details. We've also got at least a few other miscellaneous items up for grabs. ------------------------------ volunteering to help BALUG Want to volunteer to help out BALUG? Quite a variety of opportunities* Drop us a note at: balug-contact at balug.org Or come talk to us at a BALUG meeting. *e.g.: o assist on publicity o assist on speaker coordination/procurement, etc. o webmastering o archivist/history/retrieval/etc. o Linux Systems Administration (e.g. do/assist/learn, with/under some quite experienced and skilled Linux systems administrators). o chief/assistant cat herder o and other various/miscellaneous tasks BALUG "ought" to be doing or would be good to do (feel free to bring in ideas!) ------------------------------ Twitter - you can also follow BALUG on Twitter: https://twitter.com/#!/BALUG_org ------------------------------ Feedback on our publicity/announcements (e.g. contacts or lists where we should get our information out that we're not presently reaching, or things we should do differently): publicity-feedback at balug.org ------------------------------ http://www.balug.org/ From Michael.Paoli at cal.berkeley.edu Wed Mar 27 08:43:34 2013 From: Michael.Paoli at cal.berkeley.edu (Michael Paoli) Date: Wed, 27 Mar 2013 08:43:34 -0700 Subject: [buug] security/bugs - if *only* it were better Message-ID: <20130327084334.14343819mrieug84@webmail.rawbw.com> Ah, so, within the first 2 hours of my calendar day ... So, a program[0], me thinks, "Ah, that could be useful to install", but it's not part of Debian or Ubuntu ... though quite portable, signed, and also available for such (.deb). Hmmmm, me thinks, "And who wrote that software?" ... never heard of 'em. So, next I have a peek at the code. And, not too long/hard into it, yeah, security vulnerabilities. I check if they've got any bug reporting/tracking on the code - nothing (or no tracked bugs?) to be found. So, I send 'em an encrypted signed email with basic description of the issue. I also did a quick Google search on the general security issue they bumped into, and sent them 2 relevant links from the top 5 hits on Google. So, I shortly get a reply. Hey, at least they're responsive! :-) But, alas, they couldn't even decrypt the message, and even after I send it to 'em in the clear they still miss what I'm talking about. So, alas, I spell it out in more detail outlining example proof-of-concept exploit(s). Meanwhile, I also notice (and inform the above in my response to them), of those two URLs about the general issue I pulled from Google. One of which included example code of how to do it securely? Uhm, at first quick glance, yeah, but reading it more carefully and critically, nope, the example itself[1] on how to do it securely and properly, was itself not proper and secure. Ugh. But hey, it's a wiki site, I can fix that. Just sign up / register on the site and edit away. Uhm, except there's no https at all on the site, so that would all go in the clear. And their sign up / register is broken. Yes, even after I dumbed down nice secure fairly long password to nothing but alphanumerics, and removed all other data that their form might mishandle - yup, just plain doesn't work. So I email the site contact about that. (And thankfully they're also fairly responsive and email me back within several hours - and yes, they're aware of, and have been intermittently experiencing the problem, but don't have a fix ... yet). Doing security right is fairly hard. But really, it's not *that* hard. Even darn near bug free code is attainable[3]. So, yeah, found security bugs/errors in 2 completely separate sets of code within the first 2 hours of my day, and without even especially trying - those vulnerabilities did pretty much jump out at me when I read/skimmed the code. And that's not even counting issues of the wiki site having no https and a broken sign up / register page. "If builders built buildings the way programmers write programs, the first woodpecker that came along would destroy civilization." - Murphy's Laws of Computer Programming / Technology references/footnotes: 0. Maybe more details *after*[2] it's fixed. :-) 1. Maybe more details *after*[2] it's fixed. :-) 2. http://en.wikipedia.org/wiki/Responsible_disclosure 3. http://www.fastcompany.com/28121/they-write-right-stuff