Bash By Example

If you’d like to learn Bash programming, there are few resources better than the IBM DeveloperWorks site. They have an awesome series of tutorials here:

Bash By Example:

Part 1: https://www.ibm.com/developerworks/library/l-bash/
Part 2: http://www.ibm.com/developerworks/library/l-bash2.html
Part 3: http://www.ibm.com/developerworks/library/l-bash3.html

See Also:

GNU’s bash home page

Parallel Processing In Python the Easy Way!

The technical term for what we’re discussing today is Decorated Concurrency. Don’t let this scare you! 😇 It’s just a fancy (and concise) way of saying that, “We have a simple way to do parallel processing.”

Parallel processing, in turn, means that we can start multiple threads, as needed, in our program such that everything runs faster. Behind-the-scenes, on modern processors that have multiple cores, Python can run different processes on different cores. This lets a developer decide which processes will run the fastest because they have their own core. Or, at a higher level, which functions/methods are candidates for running on multiple cores.

I’ve known about parallel processing in Python for some time but it has always required making special function calls. No big deal – especially since you have to write the code to be multithreaded in the first place – but it would be nice if it was all a bit easier, right?!? Well now it is!!! 😎

The Deco (deco) project allows you to easily create code that takes advantage of parallel processing in Python. You simply do it with decorators!

If you’re new to programming, a decorator is just something that you place in your code next to a function/method/class, etc. definition. For example, the Deco project uses

@concurrent

and

@synchronized

tags. In Python you apply a decorator immediately before a function/method, etc. You can see some examples in the README.md file (that is shown to you by default) in this project: https://github.com/alex-sherman/deco (same link as above)

I expect you will be able to see some examples and some of my code very soon, as well. 😇

🙂👍

Bonus Linux tip: 

How to Isolate a CPU (core) and dedicate it to a specific task

To run a task on core 3 of a Raspberry Pi, or other Debian-based distro, including Ubuntu, simply add:

isolscpus=3

to the end of /boot/cmdline.txt. (There can be only one line in this file.)

And then run:

taskset -c 3 <your task here>

BTW, if you get a chance, please check out my new blog! http://lelandgreen.com
And if you like it, will you please share it around? 😎👍

The Art of Electronics / Learning the Art of Electronics

I have recently become the proud owner of two books, Learning the Art of Electronics (3rd Edition, Hayes/Horowitz) and The Art of Electronics (3rd Edition, Horowitz/Hill) I still love a paper book for some things, and both of these are books I’d recommend that you purchase the paper version so you can lay them open beside your projects as you work on them.

Both are excellent and I have learned a lot from them, even though I’ve barely scratched the surface. They are some 2400 pages, combined, and these are textbook-size pages. Very densely packed with information, which I want to retain. So I’m taking my time!

I think it will help us all if I can create an in-depth review, as I go. For one thing, it will help me.

An exciting quote* for today from appendix G “Where Do I Go to Buy Electronic Goodies?”:

Manufacturers Direct: Many semiconductor manufacturers (Analog Devices, TI, Maxim,…) Will not only send free samples with the slightest provocation, they will also sell in small quantities via credit card; check out Many-Circuits for RF components, and Coilcraft for inductors, transformers, and RF filters.
(*My emphasis.)

Indeed, there are too many providers and tools listed for me to include them all here. A few that sound especially interesting are octopart.comfindchips.com and netcomponents.com. You give them a part number and it searches dozens of distributors, returning (“sometimes unreliable”) information on availability and pricing.

A sneak peek of some things I intend to talk about, and some very useful shortcuts for you in the meantime!

Scan of "Learningi the Art of Electronics" pg. 14

Learning the Art of Electronics pg. 14

Scan of "Learningi the Art of Electronics" pg. 15

Learning the Art of Electronics pg. 15

 

 

Pip Boy 3000 Build Log

As I posted on my blog yesterday, my brother Noel contacted me and asked me if I would help him build a fully-functional Pip Boy 3000! I’m totally psyched to do this.

History:

Adafruit has a very detailed build here https://learn.adafruit.com/raspberry-pi-pipboy-3000/overview, and that links to the “pypboy” software here: https://github.com/sabas1080/pypboy

Update 2016.05.20: I have found there are two “main” versions of this project floating around the web. The second one being here: https://bitbucket.org/grieve/pypboy/src/92970d3a53f86eba4dc99fc979c5d8682edd2b7e?at=master I call this the “second” because it appears to have more enhancements. (And, more importantly, because I presume Adafruit linked to the original – this may be incorrect!)

Important Note: Be warned that if you use the Adafruit feature to “add all to cart” you will get a different type of knob than the project expects! I got a full-blown rotary encoder, which is a much better option is and just the “rotor-switch” (or “multi-switch) type of knob that the script now expects. (I’m working on this.)

Repository for this project: (Update 2016.05.22) I found that I can easily import the “grieve” repository into GitHub. All I did was import the project and GitHub even preserved the complete history of the project! Apparently GitHub can import from any URL, as long as it contains a Git repository. Good to know! 😎👍

So this project now has a repository located here: https://github.com/lelandg/pypboy

Update 2015.05.29 – I’ve decided to backtrack slightly. I thought that the grieve repository was newer, but I’ve found this is incorrect. The sabas1080 repository actually has the most recent commits. Not only that, there are no less than 15 forks on Github. So I am spending a little time reviewing all of these to get the most complete implementation. I will then fork that project (fork/clone). I hope to make some updates to this, too. The software appears to be well-written, but without all hardware connected, it does not function properly. At this point, I think the only “problems” may be the missing hardware. At least I want to get it connected before I start trying to “fix” anything.

Enough of that, now back to some plans…. 😇

 

Pip Boy 3000 from Fallout 3

Pip Boy 3000 from Fallout 3 and Fallout New Vegas

I have seen some builds on other sites as well. However, I have not seen a build like I plan to do! I am quite knowledgeable in Python and becoming so about the Raspberry Pi (RPi).

Since this is our hackerspace and we like to feature actual builds, I will not post details about future plans here, but rather keep this post strictly fact-based. (I always have loved a strong dose of nonfiction along with my sci-fi and fantasy desserts!)

For now, all I’ll say is this is the goal. And you can even click-through to read all about the Pip Boy 3000 on the Fallout Wiki! 😎

So far I have all of the electronics (assuming this touchscreen is big enough!) and the 3D print is scheduled to arrive Thursday, 2016.05.19! So I hope you’ll stay tuned! 

Update 2015-05-20: 3D print has arrived! This thing is awesome. Check this out:

2016.05.23: I found the py-gaugette works well almost as-is for a rotary encoder. If you want to use a rotary encoder, I found that it works well. There is another library by Bob Rathbone that is much more compact, but I found that it expected different GPIO connections than my rotary encoder has.

However… (and I apologize if you are subscribed to this post today… I’ll try to limit future updates), we have decided to go back to a rotary switch. This is actually what the script expects. After this part arrives and I get it wired-up, I’ll post some details about it, and of course a schematic when everything is finished.

2016.05.24: The rotary knob arrives today so I hope to get that soldered in place. It would be nice to complete enough of it that I could run my first test today. This may be a little optimistic, but we’ll see.

Meanwhile, I’ve already started the soldering job so I can share a photo of that progress, at least!

Several electronics parts soldered together

Soldering progress so far, along with a printout of the “schematic” from Adafruit.

Update 2016.06.25: LED Buttons!

I wanted to make the buttons function like they do in the game. However, after calls to both Mouser and Digikey, I found that there is no such thing “illuminated button” of any type that will fit inside the holes in this Pip Boy. (Drilling them larger is not an option because the 3D print contains a “lip” or flange that wraps around each LED.)

So, I had the idea to make my own using only an LED and a button. I would simply superglue the LED on top of the button and wire everything up through perfboard supporting them. Simple, right? Well,  it turns out to be much simpler in concept than in practice.

First of all the holes in the perfboard did not line up  exactly with the  holes in the Pip Boy. to solve that, I sort of tilted toward the outside two LEDs on each end. (It’s not terribly clear in the picture, but the one on the right side is a little more obvious in this picture.)

"Handmade" orange LED pushbuttons

This doesn’t give me much room on the bottom  for soldering! The two leads from the LEDs go down on either side of the switch (obviously) and then come out the perfboard just between the two leads for the button. But I’ve soldered things that were tighter than that before and got them to work, so I’m sure I can do this.

Note that the taller LEDs are required. (Some of them are shorter, and they would not fit up through the hole in the Pip Boy.) also, you don’t really have room for the  wire leads to fit up through the holes, so this might not work  for some builds.

 

References in .NET

I put together the following information while troubleshooting a nasty runtime bug that manifested only after deployment to a production environment. Since then I have returned countless times to clarify my understanding. I am posting it here in the hope that it will help others as much as it has helped me.

Introduction

In .NET, code is typically compiled into files called assemblies. It is possible for code in one assembly to invoke code in another assembly if a reference is declared. In this way, code from a large variety of sources can be combined and reused. In order for this process to work, each reference must be resolved. Reference resolution is the process of locating the concrete file corresponding to the referenced assembly. It is important to understand that reference resolution occurs at both compile time and at runtime and the process for each is totally different. Failing to understand this point can lead to endless headache. Believe me, I know.

Runtime Reference Resolution (aka binding)

When an application is invoked, it must be loaded into memory. If an application uses objects in another assembly, that assembly must also be loaded into memory. The .NET framework uses the following process to do this.

  • Determine version of referenced assembly.
    • The version of the referenced assembly is written to the applications manifest at compile time. This version will be used unless overridden via configuration.
      • application/web.config
      • Publish Policy (overrides application/web.config)
      • machine.config (overrides Publish Policy and application/web.config)
  • If assembly was previously loaded, then re-use from cache.
  • If strong-name provided, search GAC.
  • Probe
    • If codebase element specified, then use.
      • Binding failure if not found.
      • Binding failure if version, culture, or public key mismatch.
    • Search application base path. Matches by simple name and fails if first match is wrong version.
      • If no culture provided, search root then root/[assembly name]
      • If culture provided, search root/[culture] then root/[culture]/[assembly name].
      • If web/app.config specifies probing element, search paths in privatePath. Paths must be relative to application root.

For more information see http://msdn.microsoft.com/en-us/library/yx7xezcf%28v=vs.110%29.aspx.

Compile Time Reference Resolution

Compile time resolution occurs in MSBuild during the build process. MSBuild is the build engine used by both Visual Studio and TFS. Note that for ASP.NET applications, there is an extra compile step that occurs for dynamic components (aspx, asc, asax, cshtml, etc) when they are first accessed. Reference resolution for these two scenarios is described below.

MSBuild

Assembly resolution occurs in the ResolveAssemblyReferences MSBuild target. This target invokes the ResolveAssemblyReference task passing the value of the AssemblySearchPaths to the SearchPaths parameter which is assigned a value as follows.

1
2
3
4
5
6
7
8
9
10
<!-- The SearchPaths property is set to find assemblies in the following order: (1) Files from current project - indicated by {CandidateAssemblyFiles} (2) $(ReferencePath) - the reference path property, which comes from the .USER file. (3) The hintpath from the referenced item itself, indicated by {HintPathFromItem}. (4) The directory of MSBuild's "target" runtime from GetFrameworkPath. The "target" runtime folder is the folder of the runtime that MSBuild is a part of. (5) Registered assembly folders, indicated by {Registry:*,*,*} (6) Legacy registered assembly folders, indicated by {AssemblyFolders} (7) Resolve to the GAC. (8) Treat the reference's Include as if it were a real file name. (9) Look in the application's output folder (like bin\debug) -->
{CandidateAssemblyFiles};
$(ReferencePath);
{HintPathFromItem};
{TargetFrameworkDirectory};
{Registry:$(FrameworkRegistryBase),$(TargetFrameworkVersion),$(AssemblyFoldersSuffix)$(AssemblyFoldersExConditions)};
{AssemblyFolders};
{GAC};
{RawFileName};
$(OutDir)

There is a lot going on here and I don’t claim to understand all of it, but I will try to point out the important parts.

  • The most common locations to find a reference are (in search order)
    • Files added manually to project (ex. <project path>/lib/coollib.dll>
    • Location specified by hint path.
    • GAC
    • Application output path.
  • References flagged with Copy Local = true are copied to the application output path *after* compilation. This means that the value of this setting has no impact in the reference resolution process for MSBuild. Note that the Copy Local UI setting maps to the <private> element in the project file.
  • MSBuild will always try to use the latest version available for a given assembly, unless specific version = true is specified. The default value for this setting is false which means that when searching the GAC, the latest version of a DLL will always be used regardless of the version specified in the project definition.

ASP.NET Runtime Compiler

Unless previously compiled into the project output folder using the pre-compile option at build time, all dynamic content (aspx, asc, asax, cshtml, etc.) is compiled once at runtime when the application is first accessed. This dynamic content can also have dependencies on other assemblies. The system.web > compilation > assemblies element is used to tell the ASP.NET runtime compiler about these dependencies so that it can reference them.

The ASP.NET runtime compiler will search the following locations in order for these references.

  • The applications private assembly cache (aka PAC) which is the <app path>/bin folder.
  • GAC (if reference is specified using strong name).

Note that by default, the root web.config references a few system assemblies and all assemblies in the PAC using the <add assembly=”*” /> wildcard syntax. This means that you will rarely ever be required to explicitly add references manually to the system.web > compilation > assemblies element. In many cases you can and should delete the element entirely. It should only contain references to assemblies stored in the GAC. Using Copy Local = true is the recommended approach to include non-GAC references required by the ASP.NET Runtime Compiler.

Also note that many subtle errors can occur if you use the system.web > compilation > assemblies element to specify a specific version number using the assembly’s strong name. The ASP.NET runtime compiler will attempt to compile using the exact version you specify. This can cause problems if the non-dynamic components of the application were compiled against a different version of the assembly during the MSBuild compilation phase. This is often the case because MSBuild will use the latest version it can find and only the exact version if you set specific version = true.

Additional Resources:

http://jack.ukleja.com/diagnosing-asp-net-page-compilation-errors/
http://blog.fredrikhaglund.se/blog/2008/02/23/get-control-over-your-assembly-dependencies/
https://dhakshinamoorthy.wordpress.com/2011/10/01/msbuild-assembly-resolve-order/
http://www.beefycode.com/post/resolving-binary-references-in-msbuild.aspx

Unicode Emoji(!?!)

I had no idea there were Unicode emoji codes that are standardized across platforms! http://unicode.org/emoji/charts/full-emoji-list.html 🙃

There are some fun extensions for Chrome, too. I searched for “unicode emoji” and found a couple that work well, a couple that didn’t. YMMV, so I’m reluctant to recommend one of those. Just try them out. 😎

Support No Support!

Section9 is happy to announce that we have migrated to a new hosting provider called No Support Linux. For just $1/month, they provide a single website running on Apache with 1gb storage, 30gb data transfer, cPanel, SSH to a jailed root, shared hosting on fairly fast servers and no support at all!

Although we will miss Arvixe and IIS, the performance and price of No Support Linux is hard to beat.

How to Secure Your WordPress Site with CloudFlare for Free

As modern browsers and the web community in general continue to move toward secure access protocols like HTTPS, websites that do not offer these features are increasingly at a disadvantage. In the past, configuring a website to use HTTPS could be a significant challenge even for those with a technical background. Additionally the cost of purchasing the SSL certificate required for HTTPS was often prohibitive. As a result, many website owners were forced to accept the financial and technical overhead or elected not to participate in HTTPS at all.

Fortunately, the barrier to entry is much lower now. The price of SSL certificates continue to drop and organizations like Let’s Encrypt (LE) and CloudFlare provide them free of charge along with automated configuration. While I am really excited about what LE is doing, the simplicity of securing a WordPress site with CloudFlare is impressive. Simply perform the following steps.

  1. Create a CloudFlare account at https://cloudflare.com and add your website.
  2. Create a DNS entry for your domain name and ensure that the CloudFlare option is active.
  3. Ensure that the SSL configuration for your website in CloudFlare is set to Flexible.
  4. Login to your domain registrar and update your name servers to the name servers provided by CloudFlare. (Be sure to record the original values in case you want to revert back to them.)
  5. Wait for CloudFlare to request and activate a SSL certificate for your domain.
  6. Log in to WordPress and install the CloudFlare plugin.
  7. Configure the CloudFlare plugin with your domain name, API key and API email.

That’s all. You’re done! You should now be able to access your site using HTTPS.

How it works

Among other things, CloudFlare is a reverse proxy. A reverse proxy is a service that handles incoming requests on behalf of one or more websites. Requests made to a website behind a reverse proxy are actually handled by the reverse proxy and the not the originally requested website. When the reverse proxy receives a request, it contacts the requested website on behalf of the requesting client. The requested website then responds to the reverse proxy which relays the response on to the requesting client. In other words, a reverse proxy functions as a go-between for the requesting client and the requested website. This allows the connection between the requesting client and the reverse proxy to be secured with HTTPS even though the connection between the reverse proxy and the requested website is not.

This is very convenient and requires minimal change to your website; however, there are some security implications to consider. First and foremost, both you and your visitors must trust CloudFlare to be responsible and honest. Since you do not control the private keys of the CloudFlare certificates used to secure your website, you are entirely dependent upon them for your security. The keys could be lost, stolen or abused. You have no guarantees. On the other hand, CloudFlare, as far as I can tell, is a reputable and trustworthy organization and the likelihood of disaster is probably smaller than the odds of randomly running into an iceberg in the middle of the ocean with an unsinkable ship. I’m just saying.

Secondly, communication between CloudFlare and your website is, by default, no different than before CloudFlare was involved. Unless otherwise secured, information exchanged between CloudFlare and your website is not guaranteed to be confidential or unmodified in transit. In fact, there is no guarantee that CloudFlare is even communicating with your website (and not an impostor). This is perhaps even more sinister given that the person requesting your site over HTTPS has the false impression that their communication is secure when it isn’t.

For these reasons, I do believe that LE is a much better free SSL solution than CloudFlare. Unfortunately LE requires a little more technical expertise, is not fully supported on Microsoft platforms and is currently still in beta testing. So for now, CloudFlare is our top pick but stay tuned for future developments with LE.

An Introduction to SSL, TLS and HTTPS

shieldSecure Socket Layers (SSL) refers to a set of cryptographic protocols originally developed by Netscape for the purpose of securing communication between endpoints in a network. Due to security vulnerabilities, all versions of SSL have been deprecated and use of Transport Layer Security (TLS) is strongly advised. Because TLS is essentially a newer version of SSL, the term SSL is commonly used to mean either SSL or TLS.

Secure communication with a website is accomplished by means of the HTTPS protocol which is simply the use of SSL/TLS to encrypt HTTP messages. All modern browsers are capable of HTTPS communication, but it must be manually enabled on the website before it can be used.

To enable HTTPS for a website, an X.509 certificate is required. These certificates are typically purchased from a Certificate Authority (CA) such as Symantec, VeriSign, Thawte or GoDaddy and can be fairly expensive. An X.509 certificate contains information about who it was issued to (usually a website domain name), who it was issued by (usually a CA) and a public key which can be used for encryption and decryption. The public key in the certificate is mathematically related to a private key known only to the owner of the certificate. Information encrypted with the public key can only be decrypted with the private key and vice versa. This is known as asymmetric key encryption.

When a website resource is requested using HTTPS a SSL/TLS handshake must occur before any information can be exchanged. The purpose of this handshake is to verify the identify of the website, establish which cryptographic algorithms to use (the cryptosuite) and agree upon a shared master key both parties can use for encryption and decryption. In general, the process consists of the following steps. For a more detailed explanation, Chapter 4. Transport Layer Security of High Performance Browser Networking by Ilya Grigorik provides an excellent description.

  1. A TCP/IP connection is established.
  2. The browser sends information about which protocol versions and ciphersuites it supports.
  3. The server selects a protocol version and ciphersuite and attaches the website’s X.509 certificate.
  4. The browser validates the certificate, generates a master key and sends it securely to the server by encrypting it with the public key in the provided certificate.
  5. The server decrypts the master key with it’s own private key and notifies the client that it is ready to proceed.

It is worth noting that the certificate’s public key is only used once (to encrypt the shared master key in step 4 above). Although it would be possible to use the certificate’s public key to encrypt and decrypt all data sent to and from the server (eliminating the need for a shared master key altogether), it is not practical. Asymmetric key encryption is significantly slower than symmetric key encryption. Therefore, in order to maximize performance, asymmetric key encryption is used only in the handshake and symmetric key encryption is used for the remainder of the connection.

Since encryption alone can only guarantee privacy, another important aspect of the handshake is the certificate validation process. This step verifies the identity of the website and ensures that the browser is not communicating with an impostor. Certificate validation is based on a system of trust. Every X.509 certificate is signed by another X.509 certificate. This signifies that the owner of the signing certificate trusts the owner of the signed certificate. In this way, any given X.509 certificate forms a node in a chain of trust. The root certificate in every chain of trust is self-signed and must be trusted explicitly.

Typically, website certificates are leaf nodes in the trust chain and CA certificates are root nodes. Most browsers ship with a list of trusted root certificates from well known and trustworthy CA’s. Most operating systems also ship with a similar list of trusted root certificates and also provide a way for users to add new certificates to this list. In general, certificate validation in the SSL/TLS handshake simply verifies that the certificate presented by the website matches the domain name that was requested, that is has not expired or been revoked and that it chains up to an explicitly trusted root.

If any part of this process fails, the browser will inform the user that there is a problem with the certificate and may also provide an option to continue. When this occurs, there is no guarantee that the connection is being made to the intended website or that any information exchanged will be private. Although this sounds very serious, it may often be acceptable to proceed despite the warning. Ignoring a certificate validation warning is no less secure than accessing a website with the HTTP protocol (no security at all). Although it is not ideal to access any site over HTTP, it is nevertheless common practice and often the only option available. For websites that require the exchange of financial, personal or otherwise private information, a valid HTTPS connection should always be used.

In the end, the most important thing to understand about these protocols is what aspects they guarantee about communication.

  1. Confidentiality – Communication is private. This is achieved by encrypting all data with a key known only to the communicating parties.
  2. Integrity – Communication cannot be altered without detection. Although not discussed above, a Message Authenticate Code (MAC) is included in every exchange. This allows the receiver to verify that the message was not modified or corrupted in any way since the MAC was calculated.
  3. Authenticity – Communication is occurring with the intended party and not an impostor. This is verified during the certificate validation process of the SSL/TLS handshake. A fully trusted certificate implies that the owner is who they claim to be and that they (and no one else) control the certificate’s private key.

Section9 Linux tip: aliases for installing and uninstalling packages (and related)

This will be a simple post. I just wanted to share some useful things to make installing and maintaining packages much simpler (not that apt is bad).

To enable these features simply open a console window and:

sudo nano ~/.bashrc
*Note: You may not always need “sudo” to edit this file, but on some systems you may.

Scroll to the end of the file and add these lines, exactly like this:

alias install='sudo apt-get install -y'
alias purge='sudo apt-get purge -y'
alias autoclean='sudo apt-get --autoclean'
alias show='apt-cache show'
alias search='apt-cache search'

Finally, type Ctrl-X and when prompted to save type ‘y’ and then press <Enter> at the next prompt. You will then need to log off and back on, or else type:

. ~/.bashrc

BONUS TIP: Beginning a line with ‘.’ lets you run shell and startup scripts

Also be aware that you may see errors if you run your .bashrc like this. They should be safe enough, but if you experience problems, log off and back on or reboot.

You can make sure you have it setup correctly by typing:

install

And pressing <Enter>. If you get a “command not found” error then you did not get the script to execute. Go ahead and reboot in this case and it should work from then on. 🙂

This allows you to do things like:

install [someprogram]
purge [someotherprogram]
autoclean
show [package_or_program_name]
search [what_was_that_program?]

When using this search command, if you do not see any matches to your “search”, try a simpler search. You can use parts of words. And you can still type –help as a parameter to any of these new shell commands and it will show you the appropriate help screen. They really are just aliases: you still run the program they point to. In other words, for the search command, you really are running apt-cache search such that if you pass it –help, you should get help on apt-cache search features (and apt-cache in general). In this case we really are running apt-cache.


 

I hope this little introduction to using aliases for apt is instructive and/or helpful. If you think so, I hope you’ll subscribe to this blog, either with RSS or good-ol’ email. Or simply follow me (Leland Green) on Google+ or Facebook (and soon on my own web site, I hope).

Thank you for reading,
Leland…

Return top

Info

Section9 is a hackerspace based out of the Springfield Missouri area. For more information, please see the About Us page or find us on Facebook.
 
%d bloggers like this: