V4 Development Tracker

[normj]

V4 Development Tracker

This issue is for tracking the changes being made for V4 of the AWS SDK for .NET. Development of V4 is happening in the v4-development branch in this repository. V4 is an evolutionary major version change with minimal breaking changes so application code using the AWS SDK can upgrade to V4 with low effort.

Update April 28th, 2025

V4 has been released general availability. We recommend all users of the AWS SDK for .NET to upgrade to V4 to take advantage of V4's performance improvements and continue to get support for the SDK before V3 reaches maintenance mode.

https://aws.amazon.com/blogs/developer/general-availability-of-aws-sdk-for-net-v4-0/

Updated libraries that depend on the AWS SDK for .NET

The following are the list of AWS .NET libraries that have updated to V4.

• AWS.AspNetCore.DistributedCacheProvider (0.21.0-preview)
• Amazon.Extensions.S3.Encryption (3.0.0)
• AWS.Messaging (0.22.0)
• Amazon.Extensions.CognitoAuthentication (3.0.0)
• Amazon.Extensions.Configuration.SystemsManager (7.0.0)
• AWS.Logger (4.0.0)
• Amazon.AspNetCore.DataProtection.SSM (4.0.0)
• Amazon.AspNetCore.Identity.Cognito (4.0.0)
• Amazon.Lambda.DynamoDBEvents.SDK.Convertor (2.0.0)
• Amazon.Lambda.KinesisEvents (3.0.0)
• AWSSDK.SecretsManager.Caching (2.0.0)
• OpenTelemetry.Instrumentation.AWS (1.12.0)
• Aspire.Hosting.AWS (9.2.0)

Update April 25th, 2025

In preparation of V4 GA we have made some branch changes in the repository. Checkout this GitHub issue for details. #3771

Update April 9th, 2025

Preview 13 has been released that includes support for AWS service operations that require bi-directional streaming over HTTP 2. This support is available for .NET 8 and later. Examples operations are Transcribe Streaming's StartStreamTranscription API and the new Bedrock InvokeModelWithBidirectionalStream operation. The InvokeModelWithBidirectionalStream brings support to .NET Amazon Nova Sonics's speech to speech feature. The .NET Framework target of the SDK will not have the bi-directional operations due to .NET Framework not supporting HTTP 2.

At this point we consider V4 feature complete for GA. We will stay in preview mode for a little longer to collect feedback on the latest preview. We also have a bit of internal work hooking up V4 to our daily release cycle and finishing up documentation tasks. There will possibly be a few more small previews before GA for minor tweaks and fixes based on feedback.

Our ask to the AWS .NET community is to try out the latest preview of V4 to help confirm for us that V4 is ready to be GA.

Update November 11, 2024

New blog post released for preview 4 catching up some of the major changes since preview 1. https://aws.amazon.com/blogs/developer/preview-4-of-aws-sdk-for-net-v4/

Update August 15, 2024

First preview packages have been released. Blog post for the release https://aws.amazon.com/blogs/developer/preview-1-of-aws-sdk-for-net-v4/

Breaking changes

The list of breaking changes made in V4.

• .NET Framework 3.5 target has been removed.
• .NET Framework 4.5 target has been updated to 4.7.2
• Properties using value types on classes used for making requests and responses have been changed to use nullable value types. Essentially properties of type bool are changed to bool?.
• Properties using collections on classes used for making requests and responses will now default to null. The V3 behavior of initializing collections can be restored by setting the Amazon.AWSConfigs.InitializeCollections to true. This property also exists in V3 for users that want to try this behavior change before upgrading to V4.
• S3 service clients configured for us-east-1 will no longer be able to access buckets in other regions. Buckets musts be accessed with S3 service clients configured for the region the bucket is in. This change avoids surprises in applications using the S3 service client when under the covers it makes multiple AWS requests to access buckets that are not in service clients configured us-east-1 region.
• The IAM action identifiers have been removed from the Amazon.Auth.AccessControlPolicy.ActionIdentifiers namespace. These identifiers were previously marked as obsolete due to be being out of date with no mechanism to keep them up to date. Code using these ActionIdentifiers should be upgrade to use the string value of the IAM action name.
• The S3 encryption client has been removed from the AWSSDK.S3 package. This client had been marked as obsolete with the encryption client moved to its separate package Amazon.Extensions.S3.Encryption package. The following migration guide to transition to the separate package. https://docs.aws.amazon.com/sdk-for-net/v3/developer-guide/s3-encryption-migration.html
• The DetermineServiceURL method on the ClientConfig, the base class of service client configs like AmazonS3Config, was removed. This had been marked obsolete and was tied to an internal endpoint resolution system that was removed in V4. Application code calling this method should switch to using the DetermineServiceOperationEndpoint method on the service client.
• When using credential providers that rely on Amazon Security Token Service (STS) the calls will always use the regional endpoint. V3 by default used us-east-1 regardless of the configured region when running in the public partition. As part of this change the StsRegionalEndpointsValue enum was removed. Any code using that enum should be removed.
• On service client configs the RetryMode defaults to Standard instead of Legacy. The Legacy enum value was removed.
• On service client configs the DefaultConfigurationMode defaults to Standard instead of Legacy. The Legacy enum value was removed.
• DefaultClientConfig in AWSSDK.Extensions.NETCore.Setup no longer extends from service client config base class ClientConfig. The service client config properties have been replicated on DefaultClientConfig using nullable value types to allow detecting when a value has been set on DefaultClientConfig when copying the values to the service client config being created for the service client.
• CloudFront's CookieSigner and UrlSigner have been moved to a separate extension package called AWSSDK.Extensions.CloudFront.Signers to support SSL3 and take a dependency on BouncyCastle.Cryptography.
• EC2's GetDecryptedPassword has been moved to a separate package called AWSSDK.Extensions.EC2.DecryptPassword to support OpenSSL 3 and take a dependency on BouncyCastle.Cryptography.
• The SDK's source copy of BouncyCastle has been removed.
• S3's Tagging-Directive has been exposed as a public property on CopyObjectRequest and is no longer automatically set to COPY. If a user doesn't specify a tagging directive the S3 backend automatically assumes it is COPY but if a user explicitly sets it to null, then we won't set the value at all.
• AmazonS3Config's UseArnRegion will take the value set in the environment variable over that set in the ~/.aws/config file. This follows other config environment variable + config file precedence.
• Leading slashes will no longer be trimmed in CopyObject and CopyPart operations. The config option DisableTrimmingLeadingSlash has been removed.
• Removed obsolete properties such EndTime then changed EndTimeUtc to EndTime. This could lead to offset times if anyone was still using the marked obsolete original EndTime for example. A compile time error will occur for anyone using EndTimeUtc. Removing BackwardCompatibility properties and code in the generator is part of this change. Also id this to the manual S3 models.
• Response unmarshallers for TimeStamps and list TimeStamps for formats TimestampFormat.ISO8601 || TimestampFormat.RFC822 datetimes were being parsed into local times. Adjusted DateTime parsing to return UTC times. (BaseResponseUnmarshaller.tt, MultiValueHeaderParser.cs)
• Fixed the DateTimeUnmarshaller which was parsing datetime strings into and returning them as local time which in some cases were still getting converted back to UTC on a prior bug fix but not always. DateTime strings unmarshalled are assumed to be UTC time and will be specified and unmarshalled as UTC. (SimpleTypeUnmarshaller.cs)
• ConvertFromUnixEpochSeconds/ConvertFromUnixEpochMilliseconds incorrectly returning the Unix Epoch time as localtime instead of by definition is a UTC time. This changes the behavior where these methods were used. These were assigned to headers typically: https://github.com/search?q=repo%3Aaws%2Faws-sdk-net+ConvertFromUnixEpoch&type=code (AWSSDKUtils.cs)
• DynamoDB RetrieveDateTimeInUtc has been switched to true as the default.
• Removal of AWSConfigsS3.UseSignatureVersion4 and AWSConfigsS3.UseSigV4SetExplicitly to always use SigV4.
• Removal of ClientConfig.SignatureVersion only used by S3 for backward compatibility.
• Removal of Endpoint.SignatureVersionOverride from the obsolete Endpoint class used to handle overriding signature versions for S3.
• Updated AmazonS3Util.PostUpload and S3PostUploadSignedPolicy.GetSignedPolicy to use SigV4. S3PostUploadSignedPolicy.GetSignedPolicyV4 removed and overwrote to S3PostUploadSignedPolicy.GetSignedPolicy.
• Removed EC2: DiskImageImporter, ImportCleanup, EC2Metadata, and S3UploadPolicy classes.
• Support for EC2 IMDS v1 protocol has been removed. The SDK will always use IMDS v2 protocol when fetching credentials and other metadata from IMDS.

Other noteworthy changes

The following DateTime related changes were made in these two PRs:
#3572
#3601

• Fixed Epoch date to UTC per definition where the epoch date was created in local time. (TokenBucket.cs)
• Stopped using expiry times as localtime. Changed to UTC. (InstanceProfileAWSCredentials.cs, ProcessAWSCredentials.cs, RefreshingAWSCredentials.cs)
• Ensured DateTime.Max and DateTime.Min are marked with a DateTime.Kind DateTimeKind.Utc for proper calculations.
• Instead of assuming SAML credentials are localtime then converting to UTC assume that the time given is UTC to workproperly with credential expiration being in UTC time for other credential providers. (SAMLImutableCredentials.cs)
• Console logger outputs timestamps as a UTC date incase output is sent off the local machine and for easier comparison with other UTC dates. We could add a preference flag here though. (Logger.Console.cs)
• RetryPolicies return UTC server time instead of a UTC time converted to local time. (RetryPolicy.cs)
• AWSPublicIpAddressRanges mixing UTC and local time. (AWSPublicIpAddressRanges.cs)
• GetFormattedTimestampISO8601 incorrectly creating a DateTime object as local time even though it is passed in as UTC. Then formatting it as a UTC string. Although incorrect the DaeTimeKind.Local flag didn't actually harm anything and the behavior is the same by not using it.

Removed Code

• The obsolete ECSTaskCredentials class is removed. Please use the GenericContainerCredentials provider which also supports EKS Pod Identities.
• The obsolete StoredProfileAWSCredentials and StoredProfileCredentials is removed. Please use the NetSDKCredentialsFile or the SharedCredentialsFile.
• The obsolete HasCachedAccessTokenAvailable method on the SSOAWSCredentials class is removed.
• The obsolete EnvironmentAWSCredentials is removed. Please use AppConfigAWSCredentials instead.
• The obsolete StoredProfileFederatedCredentials is removed. Please use FederatedAWSCredentials instead.
• The obsolete StoredProfileSAMLCredentials class in SecurityTokenService is removed. Please use FederatedAWSCredentials instead.
• The obsolete EndpointDiscoveryEnabled class is removed. Please use the EnvironmentVariableAWSConfiguration instead.
• The obsolete UseSigV4 property on the AmazonWebServiceRequest class is removed. SignatureVersion will be used directly instead.
• The obsolete TryGetSection methods in ProfileIniFile that don't take in nested properties is removed. Please use the TryGetSection method in ProfileIniFile that accounts for nested properties.
• The obsolete Parameters dictionary in WebServiceRequestEventArgs is obsolete. ParameteCollection will be used instead.
• The obsolete DoesS3BucketExistAsync(string bucketName) in ICoreAmazonS3 is removed because it always uses HTTP. Please use Amazon.S3.Util.AmazonS3Util.DoesS3BucketExistV2Async instead.
• The following methods in AWSSDKUtils are removed ResolveResourcePath, ProtectEncodedSlashUrlEncode, ConvertToUnixEpochMilliSeconds
• EC2_METADATA_SVC, EC2_METADATA_ROOT, EC2_USERDATA_ROOT, EC2_DYNAMICDATA_ROOT, EC2_APITOKEN_URL in EC2InstanceMetadata are removed. The fields were marked as obsolete in v3.
• The obsolete ProfileManager class in Amazon.Util is removed. Please use the SharedCredentialsFile or the NetSDKCredentialsFile instead.
• The following obsolete properties on AWSConfigs are removed: Logging, ResponseLogging, and LogMetrics. Use LoggingConfig instead.
• public static Condition NewCondition(DateComparisonType type, DateTime date) in Amazon.Auth/AccessControlPolicy/ConditionFactory is removed. Please use NewConditionUtc instead.
• In the ClientConfig class, the obsolete DetermineDnsSuffix has been removed. Use the service specific client.DetermineServiceOperationEndpoint method instead. The ReadEntireResponse property has been removed, use the AWSConfigs.LoggingConfig.LogResponses or ClientConfig.LogResponse instead.
• Removal of the obsolete EventBridgeSigner class.

DynamoDB Changes

In V4 we're addressing issues around testability, improving configuring the high-level Table and DynamoDBContext clients, and potentially additional fixes that require breaking changes.

• In DynamoDB's high-level programming model, initializing the Table with a mocked IAmazonDynamoDB now returns an InvalidOperationException instead of a NullReferenceException . Async Table methods should now work with a mocked client but you may still see exceptions when calling sync methods from .NET/Core/Standard. See PR #3388, this mostly addresses DynamoDB Table misrepresents dependency on IAmazonDynamoDB #1589.
• In DynamoDB's object persistence programming model, we've separated the shared DynamoDBOperationConfig into new operation-specific objects (SaveConfig, LoadConfig, QueryConfig, etc.). The shared config has grown overtime, and contains properties that don't apply to every operation which can lead to confusion. we've marked the methods that take DynamoDBOperationConfig as obsolete. We won't remove them from V4 yet, though may do so in a future version. See PR #3421
• In DynamoDB's object persistence programming model, we've removed MetadataCachingMode and DisableFetchingTableMetadata from DynamoDBOperationConfig, and did not include these on the new operation-specific configs that were introduced above. These are table-level settings that should be specified on the global AWSConfigsDynamoDB.Context or on DynamoDBContextConfig. See PR #3422
• In DynamoDB's object persistence programming model, DynamoDBOperationConfig no longer inherits from DynamoDBContextConfig. This prevents you from passing a DynamoDBOperationConfig in to the constructor for DynamoDBContext, where some properties on the operation-specific config (such as OverrideTableName) do not apply. See PR #3422
• DynamoDBStreams has been removed from the DynamoDB package and is now in its own package and namespace.
• Community PR #3476 Allow to remove TableNamePrefix on individual operation level. Thanks Oleksandr Liakhevych
• We have added new operation specific interfaces to allow customers to mock DynamoDB operations. See PR #3450. The factory methods on IDynamoDBContext have been updated to return the new interfaces.
  • Object Persistence Programming Model
    • Mock BatchGet operations via IBatchGet and IMultiTableBatchGet interfaces.
    • Mock BatchWrite operations via IBatchWrite and IMultiTableBatchWrite interfaces.
    • Mock TransactGet operations via ITransactGet and IMultiTableTransactGet interfaces.
    • Mock TransactWrite operations via ITransactWrite and IMultiTableTransactWrite interfaces.
    • Mock Scan and Query operations via the IAsyncSearch interface.
  • Document Programming Model
    • Mock Table operations via the ITable interface.
    • Mock Scan and Query operations via the ISearch interface.
    • Mock TransactWrite operations via IDocumentTransactWrite and IMultiTableDocumentTransactWrite interfaces.
    • Mock TransactGet operations via IDocumentTransactGet and IMultiTableDocumentTransactGet interfaces.
    • Mock BatchWrite operations via IDocumentBatchWrite and IMultiTableDocumentBatchWrite interfaces.
    • Mock BatchGet operations via IDocumentBatchGet and IMultiTableDocumentBatchGet interfaces.
• The FromJson and ToJson methods on the Document object now use System.Text.Json instead of LitJson for serialization. A benefit of using System.Text.Json is this parser supports using the .NET Decimal type supporting higher precision for numeric floating point properties.
• DocumentModel and DataModel (Object Persistence Model) high level libraries have been updated to support Native AOT.
  • A limitation of the Native AOT support is nested types in .NET types used for serialization in the DataModel might go unnoticed by the trimming component of the .NET compiler. In that case you could receive an exception like System.InvalidOperationException: Type <type> is unsupported, it cannot be instantiated. The work around is to add the DynamicDependency somewhere in the code path that informs the trimmer about the dependency on the sub type. The constructor of the top level .NET type being saved is a likely place to use the constructor. The code below shows how to use the DynamicDependency attribute:

[DynamoDBTable("TestTable")]
class TypeWithNestedTypeProperty
{
    [DynamicDependency(DynamicallyAccessedMemberTypes.All, typeof(SubType))]
    public TypeWithNestedTypeProperty()
    {

    }

    [DynamoDBHashKey]
    public string Id { get; set; }
    public string Name { get; set; }

    public SubType SubType { get; set; }
}

class SubType
{
    public string SubName { get; set; }
}

• Implemented polymorphism support with object persistence high level library. implement polymorphism support for DynamoDB entries #3643
• Fixed issue where a DynamoDBEntryConversion was specified in the context but overridden if its value was different than the default in DynamoDBOperationConfig (While passing DynamoDBOperationConfig without specifying DynamoDBEntryConversion does not fallback to DynamoDBContextConfig #4030)

Logging

How to enable logging for the SDK has been reworked for V4. Logging to the console and system diagnostics works the same as V3 by setting the AWSConfigs.LoggingConfig.LogTo property to either LoggingOptions.Console or LoggingOptions.SystemDiagnostics. The LoggingOptions option for log4net has been removed along with the SDK's internal logic for using reflection to attach to in memory instance of log4net.

To include the SDK's logging into a logging framework a separate adaptor package is used to connect the SDK with the the logging framework. We have released AWSSDK.Extensions.Logging.Log4NetAdaptor for 'log4net' and AWSSDK.Extensions.Logging.ILoggerAdaptor for Microsoft.Extensions.Logging.

Example configuring Microsoft.Extensions.Logging

Add AWSSDK.Extensions.Logging.ILoggerAdaptor NuGet package and call the ConfigureAWSSDKLogging extension method from the ILoggerFactory.

var builder = WebApplication.CreateBuilder(args);

var app = builder.Build();

app.Services.GetRequiredService<ILoggerFactory>()
    .ConfigureAWSSDKLogging();

Example configuring for log4net

Add AWSSDK.Extensions.Logging.Log4NetAdaptor NuGet package and call the static ConfigureAWSSDKLogging method from Log4NetAWSExtensions.

using Amazon.DynamoDBv2;
using Amazon.Extensions.Logging.Log4NetAdaptor;
using log4net;

[assembly: log4net.Config.XmlConfigurator(ConfigFile = "log4net.config")]

Log4NetAWSExtensions.ConfigureAWSSDKLogging();
var logger = LogManager.GetLogger(typeof(Program));

---

The following changes were made in the SDK. They do not require changes to applications code using the SDK.

• The following dependencies were added to the .NET Framework 4.7.2 and .NET Standard 2.0 targets
  • System.Buffers
  • System.Memory
  • System.Text.Json
• AWSSDK.Extensions.NETCore.Setup has been made Native AOT compatible.
• For JSON based services, the SDK will now use System.Text.Json for serialization. For Non-NetFramework targets, the SDK will utilize buffer pooling for writing. For reading, the SDK will utilize buffer pooling for all targets.
• In .NET Standard 2.0, .NET Core 3.1 and .NET 8 the AssemblyVersion will match the AssemblyFileVersion. In .NET Framework 4.7.2 target AssemblyVersion will continue to use the V3 pattern of locking the version to the first 2 parts of the the AssemblyFileVersion. This is done because .NET Framework treats the AssemblyVersion as part of the identity assembly. This causes assembly collisions and forced recompilation when mixing versions of the SDK.
• The embedded endpoints.json file in AWSSDK.Core was removed. This was a 1 MB json file that was parsed at startup for the SDK to determine the service endpoint for a region. Endpoint resolution has been replaced with a new system that generates rules into each individual service assembly for determining the service endpoints.
• Community PR #3359 Optimize GetExtension execution time on .NET 8. Thanks Steven Weerdenburg
• Community PR #3293 improving AWSSDK.ToHex performance. Thanks Steven Weerdenburg
• Community PR #3292 improving AWSSDK.CopyTo performance. Thanks Daniel Marbach
• Community PR #3324 optimizing the AWS SigV4 signer. Thanks Daniel Marbach
• Community PR #3363 improving AWSSDKUtils.ParameterAsString performance. Thanks Daniel Marbach
• Community PR #3365 improving AWSSDKUtils.DetermineService performance. Thanks Daniel Marbach
• Community PR #3307 Optimizing AWSSDKUtils.UrlEncode performance. Thanks Daniel Marbach
• Community PR #3425 Avoid allocating byte[] when converting MemoryStream to String. Thanks Paulo Morgado
• Community PR #3423 Refactor user agent construction to avoid creating unnecessary string objects. Thanks Paulo Morgado
• Community PR #3439 Refactor JSON request marshaller to avoid creating unnecessary intermediate strings. Thanks Paulo Morgado
• Community PR #3458 Add ToString methods for better enum string conversion.
  Thanks Paulo Morgado
• Community PR #3570 Add possibility to add AWS services to DI with a key. Thanks Oleksii Zuiev
• Communit PR #3545 Implement Microsoft.Extensions.AI's IChatClient / IEmbeddingGenerator for IAmazonBedrockRuntime. Thanks Stephen Toub

[Dreamescaper]

Is there any way to test those packages? Some preview nuget source?

[dscpinheiro]

We'll publish the preview packages to NuGet (e.g. <PackageReference Include="AWSSDK.AccessAnalyzer" Version="4.0.0.0-preview" />), we're just working on some changes to our internal build system at the moment (we'll update this issue as soon as the packages are available).

[normj]

First preview packages have been released. Blog post for the release https://aws.amazon.com/blogs/developer/preview-1-of-aws-sdk-for-net-v4/

[martincostello]

Are you able to provide a preview version of Amazon.AspNetCore.DataProtection.SSM that depends on AWSSDK.Extensions.NETCore.Setup?

Trying to enable native AoT for a project that depends on the SSM package gives a bunch of warnings:

ILC : Trim analysis error IL2026: Microsoft.Extensions.DependencyInjection.ExtensionMethods.PersistKeysToAWSSystemsManager(IDataProtectionBuilder,String,Action`1<PersistOptions>): Using member 'Microsoft.Extensions.DependencyInjection.ServiceCollectionExtensions.TryAddAWSService<IAmazonSimpleSystemsManagement>(IServiceCollection,ServiceLifetime)' which has 'RequiresUnreferencedCodeAttribute' can break functionality when trimming application code. The AWSSDK.Extensions.NETCore.Setup package has not been updated to support Native AOT compilations. 
C:\codebuild\tmp\output\src1982900738\src\aws-sdk-net\sdk\src\Core\Amazon.Runtime\ConstantClass.cs(84): Trim analysis error IL2072: Amazon.Runtime.ConstantClass.Intern(): 'type' argument does not satisfy 'DynamicallyAccessedMemberTypes.PublicFields' in call to 'Amazon.Runtime.ConstantClass.LoadFields(Type)'. The return value of method 'System.Object.GetType()' does not have matching annotations. The source value must declare at least the same requirements as those declared on the target location it is assigned to.
  ILC: Method '[AWSSDK.Extensions.NETCore.Setup]Amazon.Extensions.NETCore.Setup.DefaultClientConfig..ctor()' will always throw because: Missing method 'Void Amazon.Runtime.ClientConfig..ctor()'
  ILC: Method '[Amazon.AspNetCore.DataProtection.SSM]Amazon.AspNetCore.DataProtection.SSM.SSMXmlRepository+<GetAllElementsAsync>d__9.MoveNext()' will always throw because: Missing method 'Void Amazon.SimpleSystemsManagement.Model.GetParametersByPathRequest.set_WithDecryption(Boolean)'

It's possible some of the above are also coming from internal libraries for transient dependencies that haven't been compiled against v4 yet either.

The app is still blocked from 0 AoT warnings due to a dependency on another third-party library that hasn't published some fixes I made yet, but otherwise this SSM package is the final hurdle.

[normj]

@martincostello I don't have a timeline yet but we would like push out preview versions of all of our high level packages like Amazon.AspNetCore.DataProtection.SSM but we have some internal work first we need to do setting up a dual preview pipeline across our repositories.