Hi Greg, I have installed the app but when ever I closed the primary service node, request from client can not automatically route to the other instance. Another issue happend when I start multiple instance of service node, the the client getting error and throw unhandle exception, but with only primary service instance running it works well. Do you have any clue what happen to my installation?

---

spotify

Hello. Need to clarify some terminology first, to avoid confusion. With config service implementations, I use in docs three terms denoting the type of service a host process is hosting:

1) Primary Service (the business service devs write to perform custom business processing).
2) Config Service (the supporting config service allowing on-the-fly reconfigurations, load balancing, failover etc). This is derived from an provided base class automatically.
3) Node Service (the supporting service that allows nodes to be aware of eachother and communicate). This is provided as a base class automatically.

So there is no concept of a "Primary Node". There are just nodes, all peers to eachother, from zero to n running at any given time. A node must be a physical machine; hosting a process implementing the config service, and exposing at minimum the Node Service; Config Service and likely (but not always) a custom Primary Service. If two nodes are two nodes arerunning, this means there are two physical computers each running a host process, and each pointing to the same repository DB (a central DB).

If only one node is running, clients cannot of course failover if this is closed, since there is no other node running. If thereare two nodes running, then failover can occurr. However, clients must be aware of both nodes. This "awareness" comes from notifications based on a pub/sub type setup, automatically. If a client process is running (and this assume the client has been generated by the Repository Create Tool--and itself implements config service against its own separate config DB), and a new backend service node is started, the new node will send a notification to the client (and any other clients running); such that it becomes part of the load balancing when making requests against a Primary Service (#1 above). There can be cases, when no client process is running to receive such notifications, that the new node will not be able to notify clients; and might not become part of its load balanced list. You can see what serversare currently active in a clients load-balance/failover list by using ConfigWeb. Login to the CLIENT config service endpoint, and look at the connections tab. This will list the backend nodes (and individual endpoints per node, so you may see a node show up more than once if it implements mulitple endpoints for a Primary Service as the StockTrader Order Processor and Business Services do.

Also, always look in the event log, both for clients and backend services. Notifications, additions, subtractions from load balance list and peer node lists are logged there; along with any errors/warnings. For the StockTrader self-host programs (for Business Services or Order Processor Service), the Config Tab shows these in real time as they occur in the Window. This is also true for any custom services you build that you choose to host in the provided Windows base class form.

If you provide more detail on exactly what nodes you have started for StockTrader, and what clients, I may be able to provide more help/info. The SOA Map view in ConfigWeb should also list all running nodes. But rememeber a node, in config service terms, is a physical machine hosting the process that implements the cionfig service, there is a 1:1 mapping between peer nodes and physical machines.

-Greg

---

Greg Leake, Microsoft

What I mean in my question is your Node Service. First on Host1, I change the configure to use self host service (both BSL and OPS). Install on the other machine (Host2) with repository DB point to the first machine. When I use WebApp one the first machine with only self host Apps on Host1 up, it works fine, however when I start self host App on Host 2, each time request direct to those instance, it return unhandle exception (I just login and logout).

---

spotify

OK. Sounds like a configuration issue of some sort on Host2. Try this to isolate issue:

0) Start, let initialize, then close the Order Processor Service on Host2, to make sure its properly closed out. Do the same on Host2 with the Business Service self-host program.
1) Start, let initialize, the Order Processor Service on Host1. Leave running for now.
2) Start Business Service self-host on Host1.
3) Login to ConfigWeb for business services (in dropdown list, make sure you pick the self-host endpoint, not IIS host endpoint).
4) Goto Connections tab, and choose Purge All Connections to this host, if any connections (red or green) are there. These would be connections to Order Processor Service.
5)Goto Edit Configuration for the Trade Business Services, and set Order Mode to Sync_InProcess for now.
6) Logout of ConfigWeb for Business Services.
7) Login to the config endpoint for your Trade Web app (on whatever machine you are running it), using ConfigWeb, for Web app (http://[host]/trade/config.svc
8) Goto Connections tab. Click Purge all connections for host, to get rid of any connections to the trade business services.
9) Goto Edit Configuration to set Access Mode to HTTP Web Service (this is the self host program).
10) Now, startup Business Services on Host2 (host1 already has running still). In the self-host window, after it initializes, click on the SOAMap button and make sure you see BOTH host1 and host2 in the cluster, and both are green. Do the same on Host1 Business Services; to make sure it shows both Host1 and Host2 in cluster as green. It is possible, especially with firewall issue or proxy/dns issue; Host1 can see Host2 but Host2 cannot see/talk to Host1. If you do not see both in cluster as green in BOTH SOAMap windows on Host1 and Host2 for self-host business service; stop here and report back to me.
11) If both host1 and host2 check out for self-host business services; you know both have good communication to their shared repositry; and both can talk to eachother so no firewall issue on node service.
12) In ConfigWeb, make sure logged in still to the TradeWeb app (/trade/config.svc endpoint); and goto Connections tab. Choose Add Connection, and choose HTTP Web Service (the self-host business service program). Enter the server name for Host1. Click OK/Add.
13) Go to Connection tab again, and make sure you see 4 (yes, 4) Green server icons; two for Host1, and two for Host2. One set is for HTTP; one for Tcp; since you have 'subscriptions' active for both for this host by default install.
14) If not all 4 green, report back what you get. If some are red; you can look in the Windows Application event log, and note error icons for the Trade Web app; with the exact exception encountered. This will indicate any connectivity issue for any active connection to Host1 and Host2 Business Services, from the Web app client.
15) If all looks good, try running the Trade Web app as a user, and seeing quotes, placing orders etc. On every login, you should see the login noted in the main window of each self host program; so if you log a user in many times (say 4 or 5); you should see some logins on Host1, and some on Host2. This shows load balancing is working, etc. You should be able to close Host1 or Host2, all requests then routed to remaining Business Service self-host; then start up self-host again, it should start to get requests again, etc. You can also try crashing Host1 or Host2 (use task mgr, end proces for Self Host Business Service EXE, or unlpug its net cable); but know by default it will take 12 consecutive failed requests to a host before it is removed from active list and added to background poll list. Until that is hit, requests may be a bit delayed as WCF (if net cable unplugged) will wait for a timeout looking for the host. This timeout and # of consecutive failed requests is configurable in ConfigWeb via config service for the trade web app.

Let me know if all this works. In this scenario, Order Processor Service is not part of the equation; I want to make sure of your basic setup (assuming 3 compters; one web app; 2 business service nodes). If all this works, report back; next step is to do similar with the order processor service; when Order Mode is set to Asyc_MSMQ.

-Greg

---

Greg Leake, Microsoft

At step 15 when I do login aganst the self host App. I still get a unhandle exception on host 2 Event log:

---

spotify

This sounds simply like Host2 is not pointing to the same StockTraderDB as Host1. Are you sure the code on both is exactly the same; assume you are using std setup with SQL Auth and you have run StockTrader setup program on both Host1 and Host2, with the secondary node install on setup you are selecting existing databases (choosing not to create new databases).

Note that with StockTrader 2.04, current release, connection string info is encrypted (during install/MSI setup program) in app.config and web.config files for all components. At runtime, for all this to be decrypted, setup/MSI needs to be run individually on each node on initial install, since a machine-specific key is used by .NET for the automatic decryption, which must be the same key used when this info isencrypted during setup/MSI.

I am not sure this is the issue; but it is apparent from your Host2 exception that HOst2 Business Services simply cannot connect to your shared database named StockTraderDB. So, I would isolate on just Host2, forget about HOst1 which works, and figure out why business services (self host or IIS host version) cannot talk to that database. Where is that database located? Is it SQLExpress or other edition of SQL 2005 or 2008? Has it been enabled for remote connections? Once you get Host2 properly talking to StockTraderDB, I think things will work as normal. This might simply mean uninstalling on HOst2, then running MSI again on HOst2 and choosing option to usethe existing database instance in the last step of setup.

-Greg

---

Greg Leake, Microsoft

Hi, I have reinstall MSI with the config to point to the existing DB. However, it still the same error. Do you have any suggestion to figure out what's wrong with the connection to tradedb. As I can see, the self host BSL start succesfully and connect to the repository. This means there is no problem with the connection to repositorydb install on HOST1 (any perhaps the tradedb as well). The parameter to connect to tradedb is in repository. Therefore I was not able to use BSL on the HOST1 if those information incorrect , which is not my case.

---

spotify

So, Host1 looks ok. Are you using SQLExpress as the repository (and StockTraderDB) on this machine, installed on this machine; or are you using a separate dedicated database server?

The trick is to install StockTrader MSI on HOST2, but instead of choosing create new database, point instead to the same database server you used with Host1. In setup, enter sa credentials, and TEST CONNECTION to make sure you have connectivity from HOST2 to the database. HOST1 and HOST2 will share the same repository, and by default that repository for business services will then provide the same database location for StockTraderDB.

I have only used SQLExpress on a single machine, with everything running on that machine. I do believe, however, that SQL Management Studio Express can be used to allow SQLExpress to accept connections from remote machines (not sure of the technical limitations/throttles set for nuimber of concurrent connections however).

If HOST2, after uninstall-reinstall (pointing to existing database location HOST1 is using, vs creating new databases); you can startup the self-host business services program; then you know for sure HOST2 can talk to this database, whereever it is located. I believe COnfig Services uses connection pools (for every node running) of 20; but I believe the StockTraderDB (set in repository) is set to 60. In reality, you can reduce this number by quite a bit (down to 15-20) without affecting perf, even under load. I say this becuase I am wondering, if you are using SQL Express, if there is some throttle to number of concurrent client connections you are hitting.

-Greg

---

Greg Leake, Microsoft

I have a stand alone .net stocktrade on host1 using sqlserver 2005 developer edition. Your mentions trick does not help much but I believe if there is any concurent problem, Host2 was not able to connect repository which is reside on host1 (I used this installation version of db at work with a few dozen of guys connect to). The strange issue happened only when the new instance of self host BSL startup on host2 and resquest direct to that instance. Would any trick could figure out or issolate the issue from code. Anyway, I will also try to locate of my installtion next week and answer to you.

---

spotify