Information Security: Output Encoding and Error Messaging

Continuing from my last post on importance of input validation in information security, I will discuss Output Encoding and Error Messaging for security with you, from the point of view of system development. Again, for the sake of ease of reading, this is structured as a set of question and answers that follow a conversation:
 

• What is output encoding?
  Output encoding is the process of converting information or instructions being outputted by a component or a service into a particular format. The underlying idea is that the output once emitted may have to go through multiple intermediate hops before reaching the final destination, and having it properly encoded leaves little scope for attacks.


• What happens due to improper output encoding?
  Improper output encoding and the lack of it is the root of implementation flaws, since data that is safe in one context may not have been encoded for safety in another context. It is important to remember that most interpreted language and structured documents formats like HTML or XML contain information in the specified structure, which can break


• What happens if the structure is not adhered to?
  When the structure us not adhered to, injection flaws can occur during the time data is passed from an interpreted structure to another system. This gives scope to vulnerabilities like XSS, CSRF attacks, and other such threat vectors.


• How to ensure that output is properly encoded?
  For proper encoding, it is important to study the structure of your output, and identify the troublesome characters used to breakout from data into structure. Delimiting characters in strings, and escape characters - used to encode dangerous/incompatible characters are high on the list. Any other delimiters introduced by the systems and contracts should also be checked for proper encoding.


• How should one proceed with encoding?
  Encoding of troublesome characters should always be done using standard library. It is a best practice to encode anything that may be dangerous in nature, i.e, anything that is not alphanumeric.


• Are the any testing strategies for proper output encoding?
  Yes, definitely. You should be thoroughly testing the output encoding using unit test validation and encoding functions. Every input field - hidden or intentional needs to be tested in the test cases. It is recommended that these tests be done on a regular basis, since downstream and upstream systems may change regularly.


• What is meant by Error Messaging?
  Typically, hackers and attackers gain additional insights based on the kind of error messages the system provides them. It is therefore important to provide only the relevant information, keeping the messaging generic enough to not impart any additional information to an attacker. For example, consider the use case of recovering an account by email. In this case, either the entered email Id may have an account, or it may not. In such cases, a system can display the message  "Instructions successfully sent to email Id" and "Account does not exist" respectively. However, a generic message on the lines of "If an account exists, instructions will be sent to mail Id" works better as it does not provide any additional ammo to the hacker.


• How do we repel attackers effectively?
  To repel attackers, give no additional ammunition, like the example above described. Attackers focus on finding exceptional conditions within the system that they can exploit, and the text of error messages helps users, developers, and attackers alike. In practice, this translates into gracefully handling all exceptions and making error message generic
  


• So what is the right time to do proper Error Messaging?
  Having the right Error Messaging is a design time problem. It is important to define error codes or exceptions that your systems or services can emit early on during the design of your components. It is equally important to define conventions on when and how errors are to be caught, and ensuring that they will always be managed via extensive code reviews. Another best practice is to ensure errors are always handled in a consistent fashion.

So far, we have covered how to make sure that a) our systems get validated inputs, b) they emit encoded outputs and c) they don't provide additional information to attackers by having the right error messaging. In the next few posts, we will look at threat models.