ProxyAndRecord does not save query-parameters, headers and body

[raghavendrabankapur]

When we try to record a request with HttpMethod as post, then we are getting error as below. Can someone please help me if I am missing anything.

I am starting server as below

string url = "http:https://localhost:1111/";
            var server = FluentMockServer.Start(new FluentMockServerSettings
            {
                Urls = new[] { url },
                StartAdminInterface = true,
                ReadStaticMappings=true,
                ProxyAndRecordSettings = new ProxyAndRecordSettings
                {
                    Url = "http:https://www.google.com/",
                    SaveMapping = true
                }
            });

Error is as below. But when I create a static mapping for the above serevr with Post as method then it works fine. Seeing the below issue only when we try to record it.

System.InvalidOperationException: Misused header name. Make sure request headers are used with HttpRequestMessage, response headers with HttpResponseMessage, and content headers with HttpContent objects.
   at System.Net.Http.Headers.HttpHeaders.CheckHeaderName(String name)
   at System.Net.Http.Headers.HttpHeaders.Add(String name, IEnumerable`1 values)
   at WireMock.Http.HttpClientHelper.<SendAsync>d__1.MoveNext()
--- End of stack trace from previous location where exception was thrown ---
   at System.Runtime.CompilerServices.TaskAwaiter.ThrowForNonSuccess(Task task)
   at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
   at WireMock.Server.FluentMockServer.<ProxyAndRecordAsync>d__11.MoveNext()
--- End of stack trace from previous location where exception was thrown ---
   at System.Runtime.CompilerServices.TaskAwaiter.ThrowForNonSuccess(Task task)
   at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
   at WireMock.Mapping.<ResponseToAsync>d__16.MoveNext()
--- End of stack trace from previous location where exception was thrown ---
   at System.Runtime.CompilerServices.TaskAwaiter.ThrowForNonSuccess(Task task)
   at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
   at System.Runtime.CompilerServices.TaskAwaiter.ValidateEnd(Task task)
   at WireMock.Owin.WireMockMiddleware.<Invoke>d__5.MoveNext()

[StefH]

Which version do you use?

And can you post the client c# code or json?

[raghavendrabankapur]

I am using the latest version. I just created a console app and trying to record the requests. I am able to record the requests for get calls. But not able to do it for post/put/delete calls.

C# code is just a console application with above code in the main method.

[StefH]

Ok. The latest code from yesterday should fix some issues with headers. However I am not sure I ever tested something else then GET... Will have to check.

[raghavendrabankapur]

I update the binaries. Looks like the issue for me is fixed. Thank you for the prompt response.

[StefH]

Please double check and close this issue if possible.

[raghavendrabankapur]

This has regressed with some other issues. Now the mappings which are made from code are not getting recognized at all.

Example

string url = "http:https://localhost:1111/";
           var server = FluentMockServer.Start(new FluentMockServerSettings
           {
               Urls = new[] { url },
               StartAdminInterface = true,
               ReadStaticMappings=true,
               ProxyAndRecordSettings = new ProxyAndRecordSettings
               {
                   Url = "http:https://www.google.com",
                   SaveMapping = true
               }
           });

  server .Given(Request.Create().WithPath("/with/get").UsingGet())
              .RespondWith(Response.Create()
                  .WithStatusCode(200)
                  .WithBodyAsJson(new { body = "from only get verb" }));

With the above code, if I run http:https://localhost:1111/with/get is returning HttpStatusCode 410 with the response of proxy url. This was working fine. This works fine when we put the same settings in the mapping json file. Also when the recording of the above happens and we update the response body, a new mapping file will gets created with the older result rather than picking it from the updated once. Even this was working well before this latest code changes. @StefH .

[StefH]

I used latest code, and used you code example above.

The json file generated looks like:

{
  "Guid": "791a3f31-6946-4ce7-8e6f-0237c7443275",
  "Title": "",
  "Priority": 0,
  "Request": {
    "Path": {
      "Matchers": [
        {
          "Name": "WildcardMatcher",
          "Pattern": "/proxy-google-test-post"
        }
      ]
    },
    "Methods": [
      "post"
    ],
    "Body": {}
  },
  "Response": {
    "StatusCode": 404,
    "Body": "<!DOCTYPE html>\n<html lang=en>\n  <meta charset=utf-8>\n  <meta name=viewport content=\"initial-scale=1, minimum-scale=1, width=device-width\">\n  <title>Error 404 (Not Found)!!1</title>\n  <style>\n    *{margin:0;padding:0}html,code{font:15px/22px arial,sans-serif}html{background:#fff;color:#222;padding:15px}body{margin:7% auto 0;max-width:390px;min-height:180px;padding:30px 0 15px}* > body{background:url(//www.google.com/images/errors/robot.png) 100% 5px no-repeat;padding-right:205px}p{margin:11px 0 22px;overflow:hidden}ins{color:#777;text-decoration:none}a img{border:0}@media screen and (max-width:772px){body{background:none;margin-top:0;max-width:none;padding-right:0}}#logo{background:url(//www.google.com/images/branding/googlelogo/1x/googlelogo_color_150x54dp.png) no-repeat;margin-left:-5px}@media only screen and (min-resolution:192dpi){#logo{background:url(//www.google.com/images/branding/googlelogo/2x/googlelogo_color_150x54dp.png) no-repeat 0% 0%/100% 100%;-moz-border-image:url(//www.google.com/images/branding/googlelogo/2x/googlelogo_color_150x54dp.png) 0}}@media only screen and (-webkit-min-device-pixel-ratio:2){#logo{background:url(//www.google.com/images/branding/googlelogo/2x/googlelogo_color_150x54dp.png) no-repeat;-webkit-background-size:100% 100%}}#logo{display:inline-block;height:54px;width:150px}\n  </style>\n  <a href=https://www.google.com/><span id=logo aria-label=Google></span></a>\n  <p><b>404.</b> <ins>That’s an error.</ins>\n  <p>The requested URL <code>/proxy-google-test-post</code> was not found on this server.  <ins>That’s all we know.</ins>\n",
    "BodyAsBytes": "PCFET0NUWVBFIGh0bWw+CjxodG1sIGxhbmc9ZW4+CiAgPG1ldGEgY2hhcnNldD11dGYtOD4KICA8bWV0YSBuYW1lPXZpZXdwb3J0IGNvbnRlbnQ9ImluaXRpYWwtc2NhbGU9MSwgbWluaW11bS1zY2FsZT0xLCB3aWR0aD1kZXZpY2Utd2lkdGgiPgogIDx0aXRsZT5FcnJvciA0MDQgKE5vdCBGb3VuZCkhITE8L3RpdGxlPgogIDxzdHlsZT4KICAgICp7bWFyZ2luOjA7cGFkZGluZzowfWh0bWwsY29kZXtmb250OjE1cHgvMjJweCBhcmlhbCxzYW5zLXNlcmlmfWh0bWx7YmFja2dyb3VuZDojZmZmO2NvbG9yOiMyMjI7cGFkZGluZzoxNXB4fWJvZHl7bWFyZ2luOjclIGF1dG8gMDttYXgtd2lkdGg6MzkwcHg7bWluLWhlaWdodDoxODBweDtwYWRkaW5nOjMwcHggMCAxNXB4fSogPiBib2R5e2JhY2tncm91bmQ6dXJsKC8vd3d3Lmdvb2dsZS5jb20vaW1hZ2VzL2Vycm9ycy9yb2JvdC5wbmcpIDEwMCUgNXB4IG5vLXJlcGVhdDtwYWRkaW5nLXJpZ2h0OjIwNXB4fXB7bWFyZ2luOjExcHggMCAyMnB4O292ZXJmbG93OmhpZGRlbn1pbnN7Y29sb3I6Izc3Nzt0ZXh0LWRlY29yYXRpb246bm9uZX1hIGltZ3tib3JkZXI6MH1AbWVkaWEgc2NyZWVuIGFuZCAobWF4LXdpZHRoOjc3MnB4KXtib2R5e2JhY2tncm91bmQ6bm9uZTttYXJnaW4tdG9wOjA7bWF4LXdpZHRoOm5vbmU7cGFkZGluZy1yaWdodDowfX0jbG9nb3tiYWNrZ3JvdW5kOnVybCgvL3d3dy5nb29nbGUuY29tL2ltYWdlcy9icmFuZGluZy9nb29nbGVsb2dvLzF4L2dvb2dsZWxvZ29fY29sb3JfMTUweDU0ZHAucG5nKSBuby1yZXBlYXQ7bWFyZ2luLWxlZnQ6LTVweH1AbWVkaWEgb25seSBzY3JlZW4gYW5kIChtaW4tcmVzb2x1dGlvbjoxOTJkcGkpeyNsb2dve2JhY2tncm91bmQ6dXJsKC8vd3d3Lmdvb2dsZS5jb20vaW1hZ2VzL2JyYW5kaW5nL2dvb2dsZWxvZ28vMngvZ29vZ2xlbG9nb19jb2xvcl8xNTB4NTRkcC5wbmcpIG5vLXJlcGVhdCAwJSAwJS8xMDAlIDEwMCU7LW1vei1ib3JkZXItaW1hZ2U6dXJsKC8vd3d3Lmdvb2dsZS5jb20vaW1hZ2VzL2JyYW5kaW5nL2dvb2dsZWxvZ28vMngvZ29vZ2xlbG9nb19jb2xvcl8xNTB4NTRkcC5wbmcpIDB9fUBtZWRpYSBvbmx5IHNjcmVlbiBhbmQgKC13ZWJraXQtbWluLWRldmljZS1waXhlbC1yYXRpbzoyKXsjbG9nb3tiYWNrZ3JvdW5kOnVybCgvL3d3dy5nb29nbGUuY29tL2ltYWdlcy9icmFuZGluZy9nb29nbGVsb2dvLzJ4L2dvb2dsZWxvZ29fY29sb3JfMTUweDU0ZHAucG5nKSBuby1yZXBlYXQ7LXdlYmtpdC1iYWNrZ3JvdW5kLXNpemU6MTAwJSAxMDAlfX0jbG9nb3tkaXNwbGF5OmlubGluZS1ibG9jaztoZWlnaHQ6NTRweDt3aWR0aDoxNTBweH0KICA8L3N0eWxlPgogIDxhIGhyZWY9Ly93d3cuZ29vZ2xlLmNvbS8+PHNwYW4gaWQ9bG9nbyBhcmlhLWxhYmVsPUdvb2dsZT48L3NwYW4+PC9hPgogIDxwPjxiPjQwNC48L2I+IDxpbnM+VGhhdOKAmXMgYW4gZXJyb3IuPC9pbnM+CiAgPHA+VGhlIHJlcXVlc3RlZCBVUkwgPGNvZGU+L3Byb3h5LWdvb2dsZS10ZXN0LXBvc3Q8L2NvZGU+IHdhcyBub3QgZm91bmQgb24gdGhpcyBzZXJ2ZXIuICA8aW5zPlRoYXTigJlzIGFsbCB3ZSBrbm93LjwvaW5zPgo=",
    "BodyEncoding": {
      "CodePage": 65001,
      "EncodingName": "Unicode (UTF-8)",
      "WebName": "utf-8"
    },
    "UseTransformer": false,
    "Headers": {
      "Date": "Wed, 25 Oct 2017 18:57:40 GMT",
      "Alt-Svc": "quic=\":443\"; ma=2592000; v=\"39,38,37,35\"",
      "Referrer-Policy": "no-referrer",
      "Connection": "close"
    }
  }
}

And this file can be use fine as a static mapping.

So can you please help me by providing the steps you take?

[raghavendrabankapur]

So now you have the above json, just update the response status code and response body to something else. Then make the same call again, ideally, you should get this request matching and the updated response should be returned. Did you give a try for the coded part of mapping??. If you run the same request again, it is creating a new mapping. It should not be creating a new mapping right if the request matches the mappings which is saved in the json @StefH

[StefH]

Ah. Maybe I understand you problem:

When WireMock is in Proxy-mode (ProxyAndRecordSettings defined), it can only be used for that purpose.

The idea is that when you are done proying & recording, you must start and use WireMock in a normal way.

[raghavendrabankapur]

Got it. But in the post call, when we pass some post data either as plain text/form data, this is not getting recorded. Is there anything I am missing?

You can try this with the above code by calling it as an API with some post body in postman.

[StefH]

When WireMock is in proxy-mode, only the response from the real service (which is proxied) is recorded in a .json file.

What you want is that in case of a body (POST,PUT,PATCH), the Request matcher is defined so that when you send exactly the same request, WireMock will only then return the recorded response.

This is not yet build. I will take a look.

[StefH]

@Dreamescaper maybe you also have a clever idea here?

[raghavendrabankapur]

@StefH did you get to see why the post body is not getting recorded in the request?

[StefH]

I have an idea why this fails. However I dont have the solution at this moment.
I will take a look when I have some time.

[raghavendrabankapur]

@StefH even the query string parameters are not getting recorded. Not sure this has been noticed.

[StefH]

Code is fixed. You can take a look here : e25c873

If this is correct, I can create a new NuGet.

[raghavendrabankapur]

I will build the solution in my local and let you know if this working fine.

[raghavendrabankapur]

I was able to build the solution and see that body and params are getting saved in the mapping file. But this fix has created few other issues. Every time it checks for an exact match in the headers list. Using C# code we cannot do a check if request header list contains required headers, then match the pattern. The same is applicable for the static mappings too.

var server = StandAloneApp.Start(new FluentMockServerSettings()
           {
               AllowPartialMapping=true,
               Port=1111,
               ProxyAndRecordSettings = new ProxyAndRecordSettings()
               {
                   SaveMapping=true,
                   Url = "http:https://www.google.com"
               },
               ReadStaticMappings=true,
               StartAdminInterface=true,
           });

           server
              .Given(Request.Create().WithPath("/with/post/bodychecktest").UsingPost().WithBody("dataa"))
              .RespondWith(Response.Create()
                  .WithStatusCode(200)
                  .WithBodyAsJson(new { body = "test" }));


           Console.WriteLine(server.Urls);

           Console.ReadKey();
           server.Stop();

In the above code, when we invoke the API with post method in postman, it always checks for all the possible headers and also in exact match pattern. Postman generated a token by itself this will keep recording a new mapping. If we remove few headers from the list, it makes an exact match to the list also. Cannot we skip this to make headers contains? @StefH

[StefH]

1]
It is indeed not possible to use it in c# code with any ***Matcher. I will take a look to add this or change the code to just generate a string pattern based matcher.

2]
I think that proxy functionality is just for getting a basic mapping. When you have the this basis autogenerated mapping, you have to tweak it and change it according to your needs.

[StefH]

1] Code has been changed into WildcardMatcher. Please take a look.

[raghavendrabankapur]

@StefH as the header/params/cookie are array of entries, do you think it will make easy if we just have exact match or pattern match for these attributes? Should we not have contains in the array kind of search, where the pattern can be considered as a match when we have that one entry of the header/params/cookies among the list. I guess this will make this tool more flexible.

[StefH]

The generated mapping is like:

"Headers": [
	{
		"Name": "Cache-Control",
		"Matchers": [
			{
				"Name": "WildcardMatcher",
				"Pattern": "no-cache"
			}
		]
	},
	{
		"Name": "User-Agent",
		"Matchers": [
			{
				"Name": "WildcardMatcher",
				"Pattern": "PostmanRuntime/6.4.1"
			}
		]
	},
	{
		"Name": "StefTest",
		"Matchers": [
			{
				"Name": "WildcardMatcher",
				"Pattern": "true"
			}
		]
	}
]

So by default WildcardMatcher is used.

If you don't want to match on User-Agent, just delete that block.
If you want a different mapping for StefTest, change it to:

"Matchers": [
	{
		"Name": "WildcardMatcher",
		"Pattern": "*"
	}
]

[StefH]

Clear for you @raghavendrabankapur ?

[raghavendrabankapur]

Looks good I guess. Let me check if it's working as the way I was expecting.

[StefH]

@raghavendrabankapur : closing this one. If you still have issues, just open a new issue.