Adding a Note\Annotation attachment as an Email Attachment

I've been trying for several hours to get this to work to no avail.  

I have a custom entity with notes enabled.  I manually added note to one of my records with a PDF attachment.  The purpose of this is that another plugin will use this attachment to include within a custom e-mail when needed.  So the only thing I need to do is pull the note with attachment and transfer the attachment to the email attachment.

I can do this numerous ways that work with no errors (the email and attachment get created correctly), however when I download the attachment from the e-mail it says the PDF is damaged.

Here is what I have tried so far, the first and simplest approach is what I think should work:

1:

myActivityMimeAttachment["body"] = myNote.GetAttributeValue<AliasedValue>("note.documentbody").Value;

2:

myActivityMimeAttachment["body"] = (string)myNote.GetAttributeValue<AliasedValue>("note.documentbody").Value;

3:

// Converting back to byte array and to filestring and back to base 64 under different formats
string stringBase64 = (string)myNote.GetAttributeValue<AliasedValue>("note.documentbody").Value;
byte[] byteFile = Convert.FromBase64String(stringBase64);
string fileString = Encoding.Unicode.GetString(byteFile); 
myActivityMimeAttachment["body"] = Convert.ToBase64String(new UnicodeEncoding().GetBytes(fileString)); // Error - damaged

I tried several different encoding types in the above example....nothing works.

Any help is appreciated!

Coty