Uploading to Azure Blob Containers: AZCopy 403 Error

The Problem

  • Azure Explorer is uploading too slow
  • AZCopy copy feature will not upload.  Error:  “RESPONSE Status: 403 This request is not authorized to perform this operation using this permission.”

Today, I was trying to upload a directory (1.3gb) of images up for blob storage for public access.   Though I often use Azure Storage Explorer for smaller uploads, it was proving impossible with this large directory.  My connection came to a grinding halt and the files where taking impossibly too long to upload.

So, upon research, I learned that AZCopy is much more efficient with larger data transfers and after success, I found this to be true.  However,  using a direct URL as per the docs was giving me a 403 error, despite me being owner of container and container seeming to have proper permissions.  So I decided to use an SAS key at end of URL (also in docs) and the apply the azcopy copy command.

Below is how I proceeded to get it working.

The Solution

Tools used:

Notes on Adding .NET Core 2.2 Identity to Existing Project

Even though all my projects are in .NET Core now, I rarely get the opportunity to use Identity because of my work with our backend Legacy system.  Recently, though, I built a very lightweight SEO Management system for one of our sites (that allows a 3rd party to tweak our page titles, meta tags, etc) and wanted to give them user access and roles.

The entire project can be found on GitHub, but below is just a running list of sites I used to get this done, noting all the troubleshooting and stupid little mistakes I did along the way.

Adding Identity to an Already Existing Project

This part was relatively easy and the Microsoft documents provided an easy enough guide.  I believe I had some issues:

CS1902 C# Invalid option for /debug; must be full or pdbonly – with the Data Migrations because I didn’t have EntityFramework installed.   Basically, all errors in this phase were not as presented – they were mostly because I was lacking packages to migrate.

To ensure I had all the right packages installed, i used: Microsoft.AspNetCore.App -Version 2.2.6

Also, in this area, I decided not to put the connection string in appsettings.json, opting …

Site 24×7 and JSONPath Example

We had an issue a couple of days ago where two points of failure caused some downtime and so I spent most of today revisiting all my monitors to give us as much warning as possible if one of our services fails.

One particular API I set up lives on the as400 (iSeries) and operates off of it’s ZendCore server in PHP.  I quickly scripted a page to verify:  connection between ZendCore PHP and db2, that the connection to that endpoint was in fact SSL, and on what port.

The PHP code is very simple and returns a JSON string that looks like this:

{
    "AS400CONNECTION": true,
    "SSL": true,
    "PORT": "633"
}

The slight problem today was figuring out how to get Site 24×7 to not just verify the link, but check the JSON values and verify it is what I need.  If not, then send the alert.  In this case, I want to verify AS400CONNECTION is true and SSL is true.

Site 24×7 suggests using a REST API monitor and then asserting the value in JSONPath.  I was completely new to this and finding a good clean example was a bit tough, hopefully this saves someone some time.  Here’s …

Making Session Variables Work in .NET Core

The Problem

As noted, I work with Legacy and often have to bring in variables from the API that must be sustained across session.. (and I’m sure there might be a better way, comment and advise!).  Where I am at now, is I query the API and bring in the variables, but how do I keep from calling these over and over?  The old solution was session variables and so, that’s where I am at.

When I started to do this on Core, the most helpful article was this (and it’s in my comments):

https://adamstorr.azurewebsites.net/blog/are-you-registering-ihttpcontextaccessor-correctly

He leads you through the basic setup of a HttpContext helper class (that I still use today) and how to configure the startup..  Today, though, I came across a problem: I was able to Set session variables, but the Get was pulling null.

The Reason

Order.  Yes, you’ll see 1000 stackflow responses about order in Configure (and I was careful to do this in that method), but now in ConfigureServices (contrary to the example, as I am now using Core 2.2?), order again comes into play:

public void ConfigureServices(IServiceCollection services)
{
    //this MUST be before add mvc or session returns null
    services.AddSingleton<IHttpContextAccessor, HttpContextAccessor>();
    services.AddMvc();

First Heroku App: Small API in PHP to Talk to .Net Core

I’ve recently been curious about switching to a time API for my time stamps and removing any dependency the app might have on the server for a timestamp.  Upon Googling I found some paid services, some free and of the free ones, I noticed one was hosted on Heroku.  I’ve heard of Heroku, but never had a reason to attempt to use it.  This was the perfect chance.

How I Created a Small “GetTime” API

First, I created a free account on Heroku, nothing special.  After verifying my email, I logged in to my Heroku Dashboard and up on the right hand corner, selected Create New App.  I named it my company-api and out popped an app.

I decided on just plain, legacy PHP and a simple DateTime string passed thru JSON encode, just to get started.  No authentication, no timezone,  just a simple spit out if a request to the site came, like this:

<?php
    header('Content-type:application/json;charset=utf-8');

    $DateTime = new DateTime();
    $currentTime = $DateTime->format("Y-m-d H:i:s");
    echo json_encode($currentTime);
?>

I created a Git repo for this brand new file and pushed it out.  Then, I went back to Heroku, Dashboard, My App and Deploy.  I selected Github as my deploy “resource” …

Share Azure Blobs with Search and Tree View

The idea behind this was to create a nice, easy UI that users can download media files they request often.  We moved it to Azure to prevent killing our on-prem bandwidth, but then I had to deal with the flat file structure, etc.  The end result was simply: a fast search of all the blobs (with link) and underneath that, a tree structure of the blobs that they can browse through.

The Set Up

First, I created a Storage container through Azure Portal, then I used Azure Storage Explorer to create a Blob Container under that storage account.  I also set read-only permissions to my blobs by right clicking the container in Azure Storage Explorer, then: Set Container Public Access Level > Public Read Access for Blobs Only.

To make this code work, I needed to setup an environment variable for my connection string.  I used the prefix CUSTOMCONNSTR_ on my variable name as it comes in handy when deploying to Azure Web Apps.  To get the connection string:  Azure Portal > Storage Account you created > Access Keys.

setx CUSTOMCONNSTR_storageConnectionString "<yourconnectionstring>"

Finally, I got a folder I wanted to share and dragged and dropped it into my …